Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Attention Is Off By One

evanmiller.org

Post reply on HN
Print view View original

321–330 of 347 posts

Re: Attention Is Off By One

#321

Post by sashank_1509 »

I really really really hate the tone of this article. What would take me one sentence to understand, took me reading 5+ paras.

And then the sneering tone of this article, sounds unprofessional and disrespectful in my opinion.

I also am pretty sure he’s wrong or at least he has to change layernorm to make this work. Attention simply does a weighted average of the Value Vectors, his change breaks that and I think will push the output closer to 0 as you stack the layers (especially considering Layer Norm). He really should do some small experiments to validate his idea first!

Re: Attention Is Off By One

#322

Post by dragonwriter »

coldtea wrote: post #319
dragonwriter wrote: post #243

Earlier quoted context omitted.

> (donna was woke before woke was a thing) Donna Haraway was born 6 years after “stay woke” in its sense as an admonition to maintain alertness to the racist context was coined. Leaving aside a debate over whether her work is a good match for “woke”, she very much cannot have been woke before woke was a thing. (Before its recent replacement of “politically correct” as the American Right’s preferred, meaning-stripped,…

> Leaving aside a debate over whether her work is a good match for “woke”, she very much cannot have been woke before woke was a thing A game of being pedantic is always welcome: She very well could have been "woke before woke was a thing", because "woke" as the parent means it in her case, refers to the modern usage (of like, 2 decades), not the original term of the 40s that might have preceeded her birth. So take t…

The 1938 sense in which it was coin3d is exactly the sense of the 1950s and the sense that got ibcreased attention circa 2000s and catapulted to attention alongside BLM (which itself was a response to the same kind of even that the art in which the phrase was coined for responded to).

The only newer sense is the American Right’s use of the term to replace “political correctness” as an empty epithet for everything and everyone it disagrees with.

Re: Attention Is Off By One

#323

Post by hgsgm »

bambax wrote: post #318
TeMPOraL wrote: post #304

Earlier quoted context omitted.

> This is striking. If true, why not try to ignore whitespace and puctuation? It is initially, but thinking about it some more, there's a lot of information packed in whitespace and punctuation choice. Scripto continua may have worked because the few readers who lived back then expected it to encode some form of legal or religious prose, but even then they could learn things from the overall shape of the document. LL…

> the few readers who lived back then expected it to encode some form of legal or religious prose The Latin literature was extremely rich, from Cicero to Tacitus, and was certainly not limited to legal information. Here's part of your comment with white space and punctuation stripped: scriptocontinuamayhaveworkedbecausethefewreaderswholivedbackthenexpectedittoencodesomeformoflegalorreligiousprosebuteventhentheycouldl…

Now do a modern structured document with sections and bullet points and logical connectives.

Re: Attention Is Off By One

#324

Post by _chaoz »

tylerneylon wrote: post #83

1. Summary The author is suggesting that we add 1 to the denominator of the softmax that is used within attention mechanisms (not the final output softmax). The softmax inside an attention unit allows it to see key/query matches as probabilities; those probabilities support a continuous-valued version of a key-value lookup (instead of 1/0 output of a lookup, we get weights where a high weight = the desired key-value…

I am a transformer, it should definitely work

Re: Attention Is Off By One

#325

Post by nextaccountic »

chessgecko wrote: post #101
gwern wrote: post #95

Earlier quoted context omitted.

He's advertising it as fixing the spiking outliers. Did your variant have those outliers beforehand?

I guess yeah I was mostly responding to Now it’s possible that softmax should be replaced wholesale, but it’s worked pretty well for the most part, except for this one wee little bug that prevents attention heads from saying nothing. So I propose a very small tweak on which I am willing to stake all future Internet claims to being correct. The tweak is so small, yet so obvious, and it’s been sitting here under everyo…

> I didn't test for outliers

Then you don't know if the approach he is advocating actually improves what he is aiming for

Re: Attention Is Off By One

#326

Post by raufer »

This is an interesting piece.

Its easy to see how for negative numbers the softmax operator could simply refrain from making a decision

e.g. ``` sum(softmax1[-100, -100, -100]) ~= 1e-43 ```

But is there any basis to assume commas and whitespaces will be negatively correlated with other tokens?

Re: Attention Is Off By One

#328

Post by numeri »

bambax wrote: post #270
nazgul17 wrote: post #200

Earlier quoted context omitted.

The way I understood it, the author is saying that, with this change, big values disappear, and we can then use fewer bits to encode the output of transformers, which means reducing the memory requirements of the network. Memory being the limiting factor to running models large, this would be a big deal.

> The Qualcomm AI researchers found that 97%+ of outlier activations in LLMs occur in whitespace and punctuation positions. This is striking. If true, why not try to ignore whitespace and puctuation? In old Latin, scripto continua [1] was a way to write continuously, for the exact same reason : to save space. Other modern languages still do that, and are no less parseable. Granted, it's unlikely a commercial LLM woul…

In the Qualcomm paper cited, they explain/hypothesize that Transformers learn to attend to these low-meaning tokens when they want to avoid adding too much extra info to the residual stream. So it's not an issue that the models attend to spaces and punctuation during in these outliers – it's the workaround the models come up with to get around the fact that attention has to go somewhere.

This post's author has a different solution, and one that theoretically could avoid causing large outliers that prevent efficient quantization. These large outliers seem to be an unfortunate side-effect of the models' learned solution.

So getting rid of spaces would do nothing to solve the problem, and would instead force the models to learn a new solution, one that presumably isn't as optimal.

Re: Attention Is Off By One

#329

Post by bambax »

hgsgm wrote: post #323
bambax wrote: post #318

Earlier quoted context omitted.

> the few readers who lived back then expected it to encode some form of legal or religious prose The Latin literature was extremely rich, from Cicero to Tacitus, and was certainly not limited to legal information. Here's part of your comment with white space and punctuation stripped: scriptocontinuamayhaveworkedbecausethefewreaderswholivedbackthenexpectedittoencodesomeformoflegalorreligiousprosebuteventhentheycouldl…

Now do a modern structured document with sections and bullet points and logical connectives.

    string.replace(/[\s\.\*\!\?,;:\-–\|"'\[\]\(\)]/g, '')

Re: Attention Is Off By One

#330

Post by nico »

saiojd wrote: post #272
nico wrote: post #35

Earlier quoted context omitted.

It would be amazing if academia started replacing papers with videos + code I want to see: an explainer of the science/ideas/experiments/hipothesis And instructions on how to reproduce the experiments/results Some YouTubers are going in this direction

Most papers already have code, and videos are very common.

Videos showing some result, but almost never a video of someone explaining the thing they are doing

When they include good videos, they really stand out

Post reply on HN