This concept is really interesting to me, I am very very new to transformers but would love to learn more about normal transformers and differential too.
Can anyone suggest any resources?
Differential Transformer
81–90 of 188 posts
Re: Differential Transformer
#82Earlier quoted context omitted.
With a single softmax you cannot predict exactly 0, but only very small numbers. When you have a large number of values to add up, this "poisons" the output with a lot of irrelevant stuff (the noise mentioned in the paper). To make things worse, low attention values will have very low gradient, thus needing a lot of weight updates to undo that kind of mistakes. On the other hand, subtracting the output of two softmax…
> predict a weight of exactly zero for some of the values Wouldn’t this be pretty unlikely, though?
Quite the opposite — if you have a long sequence only a smattering of the words will influence the meaning of the current word. Everything else is “noise”.
Attention is really good at finding this smattering of words (ie assign most weight there). But it struggles to put exactly 0 on the other words.
Re: Differential Transformer
#83I wonder what is lost here. Surely there's a trade-off... I'm wondering if there's any effect of "creativity", or ability to interpolate between concepts. Hallucination and creativity feel very related to me. I understand hallucinating as simply being misaligned with the space humans feel appropriate to interpolate between
> Hallucination and creativity feel very related to me. Why? I see them as just sampling errors. Sure a mistake can spark inspiration sometimes, but creativity is much more than mistakes. > I understand hallucinating as simply being misaligned with the space humans feel appropriate to interpolate between These language models are next-token predictors. The way the next token is predicted is by sampling a probability…
> Sure a mistake can spark inspiration sometimes, but creativity is much more than mistakes.
It looks like creativity has many steps but being able to come with novel, unprompted stuff is important, as long as you are able to discard the bullshit earlier.
"Hallucination" is only a problem if later layers (or additional networks) can't detect and remove it
Re: Differential Transformer
#84Does anyone understand why they are taking the difference between transformers instead of the sum? It seems to me that in a noise reducing solution we would be more interested in the sum, as random noise would cancel out and signal would be constructive. Of course, even if I'm right proper training would account to that by inverting signs where appropriate. Still, it seems weird to present it as the difference, espec…
I suspect that plus vs minus is arbitrary in this case (as you said, due to being able to learn a simple negation during training), but they are presenting it in this way because it is more intuitive. Indeed, adding two sources that are noisy in the same way just doubles the noise, whereas subtracting cancels it out. It's how balanced audio cables work, for example. But with noise cancelling headphones, we don't sum…
Oh! It's been a good while since I've worked in noise cancelling. I didn't know current tech was at the point where we could do direct reproduction of the outside noise, instead of just using mic arrays! That's very cool, it used to be considered totally sci fi to do it fast enough in a small headset.
Re: Differential Transformer
#85Like most things in this new world of Machine Learning, I'm really confused why this works? The analogy to noise-cancelling headphones is helpful but in that case we clearly know which is signal and which is noise. Here, if we knew why would we even bother to the noise-cancelling work?
Noise cancelling headphones are probably the wrong analogy here. The better example is the differential signalling used in professional audio and many digital signaling protocols like Ethernet, HDMI and USB. Instead of using one wire, referencing to ground, they send the signal as the difference between both wires. Both wires end up carrying the same signal with inverted polarity. Because both wires are running next…
I think when they bring up differential amplifiers they're referring more to the DSP technique of how headphone noise cancelling works but the actual electrical properties of how a differential amplifier does that muddies the message a bit.
It sort of feels closer to heterodyning and "demodulating" the signal encoded in the softmax. Those tiny little errors we're trying to denoise with this technique are almost closer to carrier waves (when encoded to softmax) than noise imo. This wouldn't get rid of noise in the training data or noise in the dimensionality of the key / value space. It's really only removing noise introduced by the process itself.
Re: Differential Transformer
#86does this not mean we should explore usage of talking heads (Shazeer et al) a bit more? https://arxiv.org/abs/2003.02436
Re: Differential Transformer
#87We empirically find that the setting λᵢₙᵢₜ = 0.8 − 0.6 × exp(−0.3 · (l − 1)) works well in practice
I wonder about the story behind that formula...
Re: Differential Transformer
#88Like most things in this new world of Machine Learning, I'm really confused why this works? The analogy to noise-cancelling headphones is helpful but in that case we clearly know which is signal and which is noise. Here, if we knew why would we even bother to the noise-cancelling work?
Don't look for an analogy, this just adds a new mathematical capability. It enables "negative attention", the network can say "I want to subtract the contribution of this token" in the attention calculation. Previously it could only reduce how much it adds.
The simple way of doing this would be to just remove the softmax or use a sigmoid instead, but in practice a softmax works better it seems.
Re: Differential Transformer
#89Like most things in this new world of Machine Learning, I'm really confused why this works? The analogy to noise-cancelling headphones is helpful but in that case we clearly know which is signal and which is noise. Here, if we knew why would we even bother to the noise-cancelling work?
I don't understand either. It seems the general idea is that they calculate attention twice, which due to random initialization might be expected to give two slightly different results. I'd have thought that what these two attention maps would have in common would be the signal, and where they would differ would be noise, so rather than subtracting them (resulting in all noise?!) what you really want is to add (so th…
The values between the groups are also going to diverge during training due to the structure of the DiffAttn equation.
The analogy I can think of is when you're paying attention to a variety of things and you actively avoid concentrating on something because it will distract you. You don't give it zero attention, you give it negative attention.
Re: Differential Transformer
#90Very clever. I like this kind of nitty-gritty detail work, and the change is small enough to be adapted easily by others. Bravo! I'm a little concerned about the last sentence of the section introduction of "2 Differential Transformer". It mentions using improvements from previous papers, but in the grammatical context, it's unclear if this improvement is added to both the normal transformer and their diff transforme…
The two other changes they mention have been widely adopted, and are included in at least some of the models they benchmark against. It seems they list them for completeness as changes to the original transformer architecture.