Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Differential Transformer

arxiv.org

Post reply on HN
Print view View original

51–60 of 188 posts

Re: Differential Transformer

#51

Post by blackbear_ »

freeqaz wrote: post #49
blackbear_ wrote: post #43

Earlier quoted context omitted.

With a single softmax you cannot predict exactly 0, but only very small numbers. When you have a large number of values to add up, this "poisons" the output with a lot of irrelevant stuff (the noise mentioned in the paper). To make things worse, low attention values will have very low gradient, thus needing a lot of weight updates to undo that kind of mistakes. On the other hand, subtracting the output of two softmax…

I'm able to follow most of what you're saying. It's unclear to me what "convex hull" means though. Also, where is each softmax happening here? For each attention head?

The convex hull of a set of points is the region "between" those points. So the convex hull of three points (that do not lie on the same line) is a triangle with those three points as vertices. If you add a fourth point inside the triangle, the convex hull remains the same, but if you add it outside then the convex hull becomes the four-sided region with those points as vertices.

In the context of standard transformer attention, each output lies in the convex hull ("somewhere between") the input values. With the modification of this paper, the input values can be scaled a little so that the output of different heads can be in different "regions" and thus do not interfere with each other (so yes to your third question, the two softmaxes are performed separately for each head).

Re: Differential Transformer

#52

Post by Majromax »

freeqaz wrote: post #49
blackbear_ wrote: post #43

Earlier quoted context omitted.

With a single softmax you cannot predict exactly 0, but only very small numbers. When you have a large number of values to add up, this "poisons" the output with a lot of irrelevant stuff (the noise mentioned in the paper). To make things worse, low attention values will have very low gradient, thus needing a lot of weight updates to undo that kind of mistakes. On the other hand, subtracting the output of two softmax…

I'm able to follow most of what you're saying. It's unclear to me what "convex hull" means though. Also, where is each softmax happening here? For each attention head?

> It's unclear to me what "convex hull" means though.

The convex hull (https://en.wikipedia.org/wiki/Convex_hull) of a set is the smallest convex shape that includes that set. Geometrically, it's what you'd get if you "shrink wrapped" the thing you're looking at: edges still protrude, but any indentations get smoothed over.

In this context, the grandparent comment is pointing out that with a traditional transformer block, the resulting computed value for a token can never "stick out" past some weighted average of the values of attended-to tokens, but this differential attention formalism allows that result.

Re: Differential Transformer

#53

Post by pizza »

freeqaz wrote: post #49
blackbear_ wrote: post #43

Earlier quoted context omitted.

With a single softmax you cannot predict exactly 0, but only very small numbers. When you have a large number of values to add up, this "poisons" the output with a lot of irrelevant stuff (the noise mentioned in the paper). To make things worse, low attention values will have very low gradient, thus needing a lot of weight updates to undo that kind of mistakes. On the other hand, subtracting the output of two softmax…

I'm able to follow most of what you're saying. It's unclear to me what "convex hull" means though. Also, where is each softmax happening here? For each attention head?

O_i = softmax(...) * V_i and softmax is between 0 and 1, so O_i = alpha * V_i for some alpha between 0 and 1 so that makes it convex, and it makes the O_i just a shrunken version of V_i. Whereas if you have the diff of softmaxes, you get O_i = (alpha - beta) * V_i, which can range from -V_i to +V_i, so its output could rescale /or/ flip V_i. And yes this is happening in every head in parallel, then they get summed.

Re: Differential Transformer

#55

Post by HarHarVeryFunny »

pikseladam wrote: post #2

Did this mean they solved the hallucination problem of transformers? edit: not fully but it gives promising results. quiet an improvement actually.

I don't think there's any narrow definition of what "hallucination" means. It generally refers to the model giving non-factual answers in contexts that are meant to be factual, but not all causes of this are going to be fixable without very major changes.

The fundamental issue is that most of the time LLMs are going to be combining statistics derived from many training samples when generating a single continuation, and there is just no guarantee that this will result in a semantically coherent response. Of course the model's depth of parsing and semantic analysis usually means that each generated word is highly plausible, but this isn't the same as being factually correct, especially so in these cases where the model is drawing on multiple sources to create a mashup response, which is the normal mode of operation.

Re: Differential Transformer

#57

Post by pizza »

Was just going to mention that it seems that it should be possible to make a Flash Attention version of this algorithm and was pleasantly surprised to see they already included an implementation of one :)

Re: Differential Transformer

#58

Post by iandanforth »

The key bit I didn't understand at first was what happens if the two groups of attention learn the same thing; because their attention masks are subtracted from one another if they both output similar values the attention across the board will drop to zero and this will lead to high loss. So the only way to reduce loss is if they learn to attend to different things. One of the simplest strategies they could learn (and this paper claims that they do) is for one group to focus on relevant context and the other to focus on irrelevant context. Thus one group learns the noise and the other the signal (it's not this cut and dry but is a useful simplification for understanding IMO).

Re: Differential Transformer

#59

Post by dartos »

> By being less distracted by irrelevant context, Diff Transformer can mitigate hallucination in question answering and text summarization

I’m very interested in this claim. I was under the impression that hallucination is unavoidable in these kinds of models. IIRC proof for that was trending on HN a couple weeks ago.

Re: Differential Transformer

#60

Post by dartos »

iandanforth wrote: post #58

The key bit I didn't understand at first was what happens if the two groups of attention learn the same thing; because their attention masks are subtracted from one another if they both output similar values the attention across the board will drop to zero and this will lead to high loss. So the only way to reduce loss is if they learn to attend to different things. One of the simplest strategies they could learn (an…

There’s probably a small chance that they could both learn the same thing, but it’s probably not likely enough to be a major issue.
Post reply on HN