Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Attention Is Off By One

evanmiller.org

Post reply on HN
Print view View original

251–260 of 347 posts

Re: Attention Is Off By One

#251

Post by lovelearning »

ersiees wrote: post #116

This trick “they found” is part of the standard torch implementation of multi head attention, namely it is called, add_zero_attention. They add a zero to the logits, resulting in a one in the denominator as e^0=1 https://pytorch.org/docs/stable/generated/torch.nn.Multihead...

I find its documentation quite poor though: "If specified, adds a new batch of zeros to the key and value sequences at dim=1."

Doesn't describe the implications even briefly. If they add just your second sentence to that description, it'll immediately become so much more useful.

Re: Attention Is Off By One

#252

Post by fogof »

Legend2440 wrote: post #40
mlsu wrote: post #8

I don't really understand the subject matter enough, so I apologize in advance for the meta-comment... The author mentions that he would maybe have written this as a scientific paper: > I tried writing a serious-looking research paper about the bug and my proposed fix, but I lost a series of pitched battles against Pytorch and biblatex, so I figured I’d just write a blog post instead. (History is written by the winne…

Counterargument: this blogpost is worthless. You get all the way to the end and then find out he hasn't actually tried it, not even on a toy model. It's just a neat idea he thinks will work.

He says in the very first paragraph:

> I lost a series of pitched battles against Pytorch and biblatex, so I figured I’d just write a blog post instead.

So I think your accusation of his burying the lede on the lack of experiment is unwarranted.

Re: Attention Is Off By One

#253

Post by fogof »

mlsu wrote: post #8

I don't really understand the subject matter enough, so I apologize in advance for the meta-comment... The author mentions that he would maybe have written this as a scientific paper: > I tried writing a serious-looking research paper about the bug and my proposed fix, but I lost a series of pitched battles against Pytorch and biblatex, so I figured I’d just write a blog post instead. (History is written by the winne…

A lot of thoughts in this thread on what academic papers are or should be, let me give my own opinion as a person who tries to write papers.

Papers should be structured like fractals - that is, they should be "self-similar". The main text of the paper after the introduction should go into all the necessary details demonstrating the origins of the idea and proving that it has value. Then the introduction section should summarize all this, and take a less rigorous tone. The abstract should be a summary of the introduction. And then the title should summarize the abstract. If you really have a lot of technical work to do, maybe you can write a super long appendix and have the main body summarize that.

I myself probably spend as much time reading paper introductions as I do reading paper bodies, which means that probably 90% of the papers I read, I only read the introduction. I do this because I enjoy it more - I like new ideas, and the intros are a great way to get a lot of them. This blog post reads like a great paper introduction to me. It's easy to trick yourself into believing something is easy though, so an academic paper would have to back this up with an experiment.

Re: Attention Is Off By One

#254

Post by danielmarkbruce »

p1esk wrote: post #220
danielmarkbruce wrote: post #150

Earlier quoted context omitted.

Are you asking "why would you have bothered to look at"? The "how" is pretty straightforward.

He's questioning the statement: "I don't think [the trick] ever did very much", because no one has yet looked at whether the trick helps reducing outliers in very large models. If it does help with this, as the blog author believes, then it is indeed a very useful trick.

Is he? A surface level reading suggests he's asking "how would you know".. and the answer is... by looking at the parameters. People do that.

>> because no one has yet looked at whether the trick helps reducing outliers in very large models

Given a softmax version doing exactly as the blog post says is baked into a google library (see this thread), and you can set it as a parameter in a pytorch model (see this thread), this claim seems off. "Let's try X, oh, X doesn't do much, let's not write a paper about it" is extremely common for many X.

Re: Attention Is Off By One

#255

Post by sheepscreek »

cycomanic wrote: post #221

The I know it's on-vogue on HN to complain about academia, but the blog post is not making a good argument. The post could have probably gotten the point across in less than 1/4 of the overall length (probably even less than 1/8th), instead the author wrapped the the post into lots of informalisms and a thinly veiled complained about academic publishing. The result of this is reflected in the discussion here, nobody…

For what it’s worth, someone pointed out that PyTorch has an optional workaround for it in the Multihead Attention API. But yes, I had to skip over 200 comments ranting off-topic that was mildly annoying (to me).

Re: Attention Is Off By One

#256

Post by gbro3n »

Vervious wrote: post #76
r3trohack3r wrote: post #55

Earlier quoted context omitted.

There was this sociologist who had written a paper for us all to read ahead of time. I started to read the damn thing, and my eyes were coming out: I couldn’t make head nor tail of it! I figured it was because I hadn’t read any of the books on the list. I had this uneasy feeling of “I’m not adequate,” until finally I said to myself “I’m gonna stop, and read one sentence slowly so I can figure out what the hell it mea…

Systems research papers do not represent all research papers out there, not even in computer science. In cryptography, certainly a paper with formal definitions and proofs can be much more valuable than a corresponding blog post. It's a field where formalism is desired, if not necessary. Otherwise you can't check other people's "proofs", or even know what model you're working in. I think, since people haven't come up…

I don't know that much about AI, but my experience in other areas has shown me that 'more grown up' literature that feels harder to parse when your starting out later becomes the precise technical information you need as you get deeper into a subject. Like W3Schools when you start out in web dev vs MDN when you're skills are more mature.

Re: Attention Is Off By One

#257

Post by notanothereric »

If the author just proposed his new function neo_max() and quoted The Architect from The Matrix everyone here would be enjoying the April Fools Day joke for 2024…

“ Your life is the sum of a remainder of an unbalanced equation inherent to the programming of the matrix. You are the eventuality of an anomaly, which despite my sincerest efforts I have been unable to eliminate from what is otherwise a harmony of mathematical precision. While it remains a burden assiduously avoided, it is not unexpected, and thus not beyond a measure of control. Which has led you, inexorably, here.”

Hence ergo - you are a NaN. A divide-by-zero.

Re: Attention Is Off By One

#258

Post by tangjurine »

tylerneylon wrote: post #83

1. Summary The author is suggesting that we add 1 to the denominator of the softmax that is used within attention mechanisms (not the final output softmax). The softmax inside an attention unit allows it to see key/query matches as probabilities; those probabilities support a continuous-valued version of a key-value lookup (instead of 1/0 output of a lookup, we get weights where a high weight = the desired key-value…

> Statistically, we expect that averaged vector to be close to zero.

I'm not sure that's the case, especially in high dimensions.

The expected value of the absolute value n random variables, uniform [-1,1], grows with n. I'm pretty sure it's proportional to the sqrt of n.

Also, random walks in high dimension return to zero with probability zero, so the sum of random variables in high dimensions going close to zero seems unlikely as well.

Re: Attention Is Off By One

#259

Post by quickthrower2 »

jablongo wrote: post #239
tylerneylon wrote: post #83

1. Summary The author is suggesting that we add 1 to the denominator of the softmax that is used within attention mechanisms (not the final output softmax). The softmax inside an attention unit allows it to see key/query matches as probabilities; those probabilities support a continuous-valued version of a key-value lookup (instead of 1/0 output of a lookup, we get weights where a high weight = the desired key-value…

Yea - the way we can "tell if this is good" is by a) train two identical models on a large dataset, one with the +1 in the denominator for the softmax steps of the attention modules, one without b) show that they have similar performance (doubt the +1 will make performance better, but we need to show it doesn't make things worse) c) show that there are less "blowups" in the model with +1, and therefore they are more…

That is a good start. I wonder though if the change affects the ideal hyperparameters. Do you need more or less dropout if you make the change? What about learning rate?

So you might want to re-search the hyper params for a fair shot.

Re: Attention Is Off By One

#260

Post by zamalek »

AlanSE wrote: post #240
nazgul17 wrote: post #200

Earlier quoted context omitted.

The way I understood it, the author is saying that, with this change, big values disappear, and we can then use fewer bits to encode the output of transformers, which means reducing the memory requirements of the network. Memory being the limiting factor to running models large, this would be a big deal.

Yeah, good to bring it back to the original point. Reading the article felt exciting, but in hindsight I am now missing a key detail. The equations all seem to be matrix operations with a fixed number of rows / columns (you can take me as a real layman here). Unless you change that, I don't understand _how_ you can reduce memory needs. Granted, I'm probably putting my foot in my mouth not understanding transformers.

More ELI5 than the other comments. Considering the softmax network:

During quantization we find that values in the network vary from 0->5000, but 95% of values are This is where exotic encodings come into play. We might try to use a logarithmic scheme, for example. This would result in higher value densities at lower values - but we would probably still waste bits and it would require more APU cycles.

Now switch to the softmax1 network:

The range of values is less important than the distribution - instead of 95% of the values falling in a small range, we would see the values more evenly spread out. Assuming that the range is now 105 (so the 5% outlying neurons from the softmax network are still >100), we would have 243 values to represent everything under 100. The same example with 19 and 30 would result in 19.27 and 30.34 respectively, a difference of 11.07 - which is very close to the unquantized difference of 11. We have retained more information in the quantized version of the network.

Information is lost either way, but what's important is how much information is lost.

The reason that the large values appear is because the heads attempt to "scream really loud" when they are certain that they are right. This is an emergent behavior due to softmax - it ironically sucks at paying attention to a few of the heads: it boosts the volume of the heads that are trying to abstain, and mutes the volume of the heads that are trying to vote.

Post reply on HN