Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Attention Is Off By One

evanmiller.org

Post reply on HN
Print view View original

101–110 of 347 posts

Re: Attention Is Off By One

#101

Post by chessgecko »

gwern wrote: post #95
chessgecko wrote: post #7

I ran an experiment like this and in my setting it didn't help. Not saying there may not have been a bug or something, but I think attending over the current position sort of solves this problem. IE when it should not speak it just emits the current pos value. edit to add details in case anyone is interested I didn't add one to the softmax denom. I added a learned parameter (the attention sink) that would be appended…

He's advertising it as fixing the spiking outliers. Did your variant have those outliers beforehand?

I guess yeah I was mostly responding to

Now it’s possible that softmax should be replaced wholesale, but it’s worked pretty well for the most part, except for this one wee little bug that prevents attention heads from saying nothing. So I propose a very small tweak on which I am willing to stake all future Internet claims to being correct. The tweak is so small, yet so obvious, and it’s been sitting here under everyone’s noses ever since attention was invented (2014).

I didn't test for outliers, but I don't think this will lead to a large improvement in attention overall/it will fix a lurking bug.

Re: Attention Is Off By One

#102

Post by janalsncm »

nico wrote: post #35
mlsu wrote: post #8

I don't really understand the subject matter enough, so I apologize in advance for the meta-comment... The author mentions that he would maybe have written this as a scientific paper: > I tried writing a serious-looking research paper about the bug and my proposed fix, but I lost a series of pitched battles against Pytorch and biblatex, so I figured I’d just write a blog post instead. (History is written by the winne…

It would be amazing if academia started replacing papers with videos + code I want to see: an explainer of the science/ideas/experiments/hipothesis And instructions on how to reproduce the experiments/results Some YouTubers are going in this direction

+1 to including code with your paper. It improves reproducibility and transparency. There’s even a well-known website dedicated to this purpose.

For the rest of it I don’t care. As long as researchers understand what’s going on, that’s what matters.

Re: Attention Is Off By One

#103

Post by Vervious »

renonce wrote: post #87
Vervious wrote: post #76

Earlier quoted context omitted.

Systems research papers do not represent all research papers out there, not even in computer science. In cryptography, certainly a paper with formal definitions and proofs can be much more valuable than a corresponding blog post. It's a field where formalism is desired, if not necessary. Otherwise you can't check other people's "proofs", or even know what model you're working in. I think, since people haven't come up…

Requiring formalism does not preclude attaching an informal but intuitional description of the formal definition or proof. Unless the authors don't understand very clearly what they are talking about, or they want to prevent others from understanding their concepts too easily, I don't see why there is a reason for the authors not to attach an EIL5 in addition to formalism.

Sure. But it's an ELI5 "in addition to formalism", not "in lieu of formalism". In theory conferences like STOC or FOCS, the first section of the paper often comprises such an overview.

Certainly some papers are better written than others. But sometimes a blog post cannot replace a paper, unless it also goes into the depth and detail that formalism requires. (Then it becomes a 30 page blog post, where most people don't read past the intro.)

Re: Attention Is Off By One

#104

Post by phkahler »

A couple thoughts. 1) An alternative might be to have an extra NULL output where the attention can be diverted. This might be what existing models are using commas for, but make it explicit. 2) What he proposes has a similar effect on the other weights without explicitly having the NULL present. In this light it should work, but does it have the advantage he thinks?

Re: Attention Is Off By One

#105

Post by knewter »

Legend2440 wrote: post #72
joebiden2 wrote: post #71

Earlier quoted context omitted.

You seem to really disregard the positions of this author. They seem to have invested substantial efforts in that specific area of research. To validate the idea the author has, it would be required to train a LLM from zero. If the author is right, you would get similar results to the current generation of LLMs, but with (a lot) less space required for the intermediate layers. The time to achieve that is still measur…

You don't need to train a ChatGPT-sized LLM, a toy nanoGPT would have been enough. You can train those on a consumer GPU in an afternoon. And yes I do disregard his research effort. There are hundreds of well-justified and well-researched "clever tricks" for improving Transformers, and almost all of them don't work. I'll believe it when I see the results.

Google used it in flaxformers since 2021 apparently

Re: Attention Is Off By One

#106

Post by hawkice »

seydor wrote: post #94

Can i please ask why lim{x->-inf} softmax(x) = 1/k ?

It's splitting the probability across all x_i equally, even when they're all massively negatively weighted.

This change would have all the probability going into the null option, in that case, basically.

Re: Attention Is Off By One

#107

Post by refulgentis »

jrochkind1 wrote: post #61

While not about AI or the algorithm mentioned, on the subject of little errors that you can't convince anyone are errors.... In 2011, I wanted to copy the reddit ranking algorithm in a project of my own, so I went to source code to look at it... the algorithm in the source code I found wasn't doing anything at all sensible with negative-sum voted posts. I thought I discovered the error, some terms swapped in the simp…

I work at a FAANG and it was absolutely astonishing to find out how often this happens.

You can make a long, impactful career by just being "the guy who adds log statements throughout the codebase and reasons through it", doing this at even a simplistic level has always shown me an astonishing fix to some long-standing issue.

n.b. It also attracts a ton of political fun. People's first order reaction is denial, and it only gets worse from there. Absolutely no one except 1-2 colleagues will see it as "oh we should fix that", and at least one person will make sure your boss' boss' boss is CCd on an email with a nice version of "no he's just insufficiently concerned about {concurrency, memory management, take your pick}" Just wait it out quietly when that happens, do not engage or complain. If nothing happens and you're never asked about it by leadership, but your peers ask, make plans to move onto another team.

Re: Attention Is Off By One

#108

Post by [deleted] »

renonce wrote: post #87
Vervious wrote: post #76

Earlier quoted context omitted.

Systems research papers do not represent all research papers out there, not even in computer science. In cryptography, certainly a paper with formal definitions and proofs can be much more valuable than a corresponding blog post. It's a field where formalism is desired, if not necessary. Otherwise you can't check other people's "proofs", or even know what model you're working in. I think, since people haven't come up…

Requiring formalism does not preclude attaching an informal but intuitional description of the formal definition or proof. Unless the authors don't understand very clearly what they are talking about, or they want to prevent others from understanding their concepts too easily, I don't see why there is a reason for the authors not to attach an EIL5 in addition to formalism.

[deleted]

Re: Attention Is Off By One

#109

Post by feoren »

ks2048 wrote: post #99

Interesting read. As others have said, it will be much more convincing with some experimental numbers. I'm confused what his goal is though: I could imagine some theoretical reason to add a 1 there, but he starts by saying this can lead to smaller, more compactable models. Is he talking about the size the compressed weights? or pruning to a smaller model? or resistant more quantization? Parts of the essay seemed to t…

He says this in the article: if you ever want to jam a multi-trillion-parameter model into a phone app or a Raspberry Pi, you must quantize. I've seen some quantization go from doubles to bytes (64 bits to 8) per weight, reducing the RAM requirement by 8x. A simple quantization (I'm sure there are much better ones) is to round everything the nearest 1/255th of your number range, then multiply by 255. So your resolution is (max-min)/255. You also store the min and max so you can reverse it, of course. Say you're trying to quantize these sets of numbers:

1. { -1.4, 0.8, 2.7, 7.3 } : With a range of 8.7, you have a resolution of 0.034. This set quantizes to { 0, 64, 120, 255 }.

2. { -1400, 800, 2700, 7300 } : Resolution 34.1, quantizing to the same as the above { 0, 64, 120, 255 }.

3. { -0.008, -0.001, 0.009, 0.019 } : resolution 0.000106. This set quantizes to { 0, 66, 161, 255 }.

3. { -1.4, 0.8, 2.7, 7329 } : Resolution 28.7. This set quantizes to {0, 0, 0, 255 }. Oops -- we can no longer tell most of our weights apart.

You can see how this quantization works really well when all the numbers are close together, regardless of their absolute scale. Major outliers completely mess up the entire system. You can make more and more complicated quantization algorithms, but those will always come with tradeoffs. The best option would be to tame your weights so that they are again close together.

Re: Attention Is Off By One

#110

Post by toxik »

sedael wrote: post #62
fwlr wrote: post #24

The author identifies a real problem and poses a simple solution. It passes all my crank tests (why did no one come up with this before? Because the author is intimately familiar with the softmax function from work outside of ML, and plausibly nobody who’s investigating these issues is remotely as familiar, so despite researchers narrowing the issue down to “something to do with softmax”, they don’t have a deep enoug…

>why did no one come up with this before So it turns out someone did. Specifically google did. This exact same idea has been in flaxformers since at least November 2021. https://github.com/google/flaxformer/blame/ee62754ebe5a5eeb1... Specifically to save people a click it says: > """Softmax function with an additional virtual logit equal to zero. For compatibility with some previously trained models. This is equivale…

Or maybe it doesn’t really do anything to improve performance.
Post reply on HN