Live data from Hacker News

Attention Is Off By One

evanmiller.org

51–60 of 347 posts

Re: Attention Is Off By One

#51
post #23

OP is right in that his change would make the softmax in the attention output zero if it "has nothing to add" (QuietAttention, as he said). Buuut, it's missing the forest for the trees. The goal of the last step of attention (ref., Fig. 2, left in https://arxiv.org/abs/1706.03762 ) is not to add/say anything (as the author is saying) but to compute the relationship between the tokens (QK^T) and V -- in layman terms,…

What's wrong with unscaled matrix multiplication? Softmax has some kind of intuition in the context, but why not layer norm or something else instead (if anything is needed at all)?

Re: Attention Is Off By One

#54
post #24

The author identifies a real problem and poses a simple solution. It passes all my crank tests (why did no one come up with this before? Because the author is intimately familiar with the softmax function from work outside of ML, and plausibly nobody who’s investigating these issues is remotely as familiar, so despite researchers narrowing the issue down to “something to do with softmax”, they don’t have a deep enoug…

> why did no one come up with this before? Because the author is intimately familiar with the softmax function from work outside of ML, and plausibly nobody who’s investigating these issues is remotely as familiar

I doubt that is true. Softmax is extremely well understood within the ML community. It's a very common trick, these properties are well-known as well. It feels very unlikely that nobody has thought of this before. That said, it's also plausible that the current softmax convention was chosen by accident and the author is right to identify this drawback.

Re: Attention Is Off By One

#55
post #8

I don't really understand the subject matter enough, so I apologize in advance for the meta-comment... The author mentions that he would maybe have written this as a scientific paper: > I tried writing a serious-looking research paper about the bug and my proposed fix, but I lost a series of pitched battles against Pytorch and biblatex, so I figured I’d just write a blog post instead. (History is written by the winne…

Nah, scientific papers are supposed to be precise and technical. This reads like those quite frequent suggestions here of switching all equations in papers to plain English or code: it honestly comes from a place of ignorance, and I say that as basically a layman myself. What should be encouraged is for academics to blog about their research as well. It would even help when recruiting and onboarding new members. Righ…

    There was this sociologist who had written a paper for us all to read ahead of time. I started to read the damn thing, and my eyes were coming out: I couldn’t make head nor tail of it! I figured it was because I hadn’t read any of the books on the list. I had this uneasy feeling of “I’m not adequate,” until finally I said to myself “I’m gonna stop, and read one sentence slowly so I can figure out what the hell it means.”
    
    So I stopped-at random-and read the next sentence very carefully. I can’t remember it precisely, but it was very close to this: “The individual member of the social community often receives his information via visual, symbolic channels.” I went back and forth over it, and translated. You know what it means? “People read.”
    
    Then I went over the next sentence, and realised that I could translate that one also. Then it became a kind of empty business: “Sometimes people read; sometimes people listen to the radio,” and so on, but written in such a fancy way that I couldn’t understand it at first, and when I finally deciphered it, there was nothing to it.

  -- Feynman
I disagree. After going through quite a few research papers in my time, I've found the best are the ones that are direct and to the point. Many papers I've spent many hours/days trying to unravel just to realize the concepts were straightforward, not very novel, and there wasn't much of real substance to the paper.

Meanwhile, some of the most impactful papers I've read are direct and to the point. Kadmellia, Bitcoin, BitTorrent, DynamoDB, Firecracker, etc.

It seems like, when you have something of substance to say, you say it. When you don't you overcompensate by falling back on building an intricate puzzle of jargon and convoluted equations in an attempt to make what you're saying sound far more important than it really is.

As LLMs get better, I look forward to the day where every journal has a standard LLM filter you're required to apply to your paper that unravels all of this nonsense and rewrites it a more straightforward way, if not to directly publish than just for the editors to verify there isn't a simpler way to convey your ideas. I suspect that if we had an EIL5 filter for most journal articles, we'd discover that a majority of the words that get published have very little substance at all.

Re: Attention Is Off By One

#56

Earlier quoted context omitted.

Then again, if you don't have access to giant compute clusters you can't test this, so it's either a blog post or nothing. I believe the outlier problem that this solves only appears for very large models.

That isn’t true at all. Train a smaller model on a smaller dataset. You can even train on your laptop. It’s definitely feasible. This is just a proof of concept, it doesn’t need to beat state of the art.

Maybe I edited my comment too late.

Re: Attention Is Off By One

#57
post #44

Don't transformers typically have a token at the beginning of the prompt? This seems equivalent to letting the network attend to this token, and produce a zero value if that's what it wants.

Chat-tuned ones do, but the base models don't. For example, Llama doesn't, but Alpaca has "### Instruction:", "### Input:", and "### Response:".

Base LLaMA still has dedicated tokens for beginning/end of string. What you're describing is the instruction format, which is separate.

Re: Attention Is Off By One

#58
I thought everyone knew that softmax (and specifically exp functions in it) are poison. I have always worked around them, for example by using large epsilons (approaching one actually), and using low-order polynomial approximations for the exp functions.

I thought everyone does that, because you don't need to work long with these models to get NaNs, and when you check why you see it's because of the exp functions. Then you fix it. Apparently people don't.

It's not like the neural models care if you approximate functions. They couldn't care less actually.

Re: Attention Is Off By One

#59
post #8

I don't really understand the subject matter enough, so I apologize in advance for the meta-comment... The author mentions that he would maybe have written this as a scientific paper: > I tried writing a serious-looking research paper about the bug and my proposed fix, but I lost a series of pitched battles against Pytorch and biblatex, so I figured I’d just write a blog post instead. (History is written by the winne…

Counterargument: this blogpost is worthless. You get all the way to the end and then find out he hasn't actually tried it, not even on a toy model. It's just a neat idea he thinks will work.

Why would that make it worthless?

Re: Attention Is Off By One

#60
post #24

The author identifies a real problem and poses a simple solution. It passes all my crank tests (why did no one come up with this before? Because the author is intimately familiar with the softmax function from work outside of ML, and plausibly nobody who’s investigating these issues is remotely as familiar, so despite researchers narrowing the issue down to “something to do with softmax”, they don’t have a deep enoug…

Yeah, but it lacks the most important test: results. He hasn't actually tried it, he just thinks it will work. For such a simple change to the softmax it wouldn't take long to verify. It's really embarrassing to not do that before publishing.

It's not embarrassing at all.

I think there might be some curse of the auto-didact here, hinging on the meaning of publish: it would be embarrassing if he was capital-P publishing, as in a scientific paper.

The blog goes to great lengths to point out it is _not_ capital-P publishing.

Post reply on HN