Attention Is Off By One
evanmiller.org
Attention Is Off By One
1–10 of 347 posts
Re: Attention Is Off By One
#2[deleted]
Re: Attention Is Off By One
#3Don't transformers typically have a token at the beginning of the prompt? This seems equivalent to letting the network attend to this token, and produce a zero value if that's what it wants.
Re: Attention Is Off By One
#4This is right below the "Have Attention Spans Been Declining? – Yes, 65%" post, lol brilliant. In general, human decreasing, AI increasing- attention.
Re: Attention Is Off By One
#5I don't understand this well enough to say if it is correct, but I do understand it well enough to say it is important if correct.
Re: Attention Is Off By One
#6This seems very important if accurate.
Re: Attention Is Off By One
#7I ran an experiment like this and in my setting it didn't help. Not saying there may not have been a bug or something, but I think attending over the current position sort of solves this problem. IE when it should not speak it just emits the current pos value.
edit to add details in case anyone is interested
I didn't add one to the softmax denom. I added a learned parameter (the attention sink) that would be appended to the beginning of QK but would be removed after softmax, so when multiplying by V the totals wouldn't sum to one. I tried variants that included looking at the current pos and not, and also variants that predicted used an ffn to generate the sink per position instead of a learned param. In my setting neither approach really made much of a difference. But I also had a bunch of other weird stuff in there too, so it may be worth trying again.
Re: Attention Is Off By One
#8I don't really understand the subject matter enough, so I apologize in advance for the meta-comment...
The author mentions that he would maybe have written this as a scientific paper:
> I tried writing a serious-looking research paper about the bug and my proposed fix, but I lost a series of pitched battles against Pytorch and biblatex, so I figured I’d just write a blog post instead. (History is written by the winners; blogs are written by…)
Honestly, thank god he didn't. This paper is so much more readable and approachable than what gets published in "serious" journals. The tone is self-effacing, it does not have an "ego" the way scientific papers tend to have. If all science read like this, and if we were "allowed" to cite research that reads like this, I think we would be much better off. This reads like a conversational, approachable textbook, not like an impenetrable wall.
Is it because I don't understand attention at a PhD level that I hold this opinion? Maybe. Could he be writing like this because he's a layman and utterly wrong about the topic, unlike those Serious Science Authors? Maybe, I don't know.
But my god, wouldn't it be nice to be allowed to write like this?
Re: Attention Is Off By One
#9I don't see any results, it'd be more impactful and convincing if there were numbers supplementing the theory. It's not that hard to finetune existing LM on a small data and verify that it works.
I am, however, of the similar opinion that there could be better attention formulations. A paper from 2020 https://arxiv.org/abs/2005.09561 helped a lot in one of the transformers model I trained (not a vanilla LM but a specialised multi-modal graph problem).
It proposes normalised attention which if I'm not wrong should help with the quantisation problem too.
Re: Attention Is Off By One
#10I don't really understand the subject matter enough, so I apologize in advance for the meta-comment... The author mentions that he would maybe have written this as a scientific paper: > I tried writing a serious-looking research paper about the bug and my proposed fix, but I lost a series of pitched battles against Pytorch and biblatex, so I figured I’d just write a blog post instead. (History is written by the winne…
I think maybe its because he didn't have experimental results that show that it worked. Not a knock against the author, there are just so many things that seem like good ideas that don't end up working well in practice, a paper like this without results is hard to value.