31–38 of 38 posts
Seems like this was a better solution to the same problem https://www.evanmiller.org/attention-is-off-by-one.html
Did this end up working? It sounds plausible but it needs some empirical validation.
Implementation for gpt-oss this week showed 2-3x improvements https://github.com/ggml-org/llama.cpp/pull/15157 https://www.reddit.com/r/LocalLLaMA/comments/1mkowrw/llamacp...
Seems like this was a better solution to the same problem https://www.evanmiller.org/attention-is-off-by-one.html
This all reminds me of the bias term of a perceptron.
And with transformers we started with not having any and the network repurposed one of the inputs for that which annoyed people because now dropping this particular input makes the whole thing be unreasonably affected but also annoyed some other people because weight on that input was unreasonably high because it sort of balanced all others.
So initially people (from hanlab) tried to affix this input so it doesn't get dropped. Then they (from openai this time) decided to just skip the input by providing learnable bias inside of the network (doing the thing that classical perceptron does) and now this guy proposes further optimization just by setting bias to 1 everywhere, which might work perfectly fine since we don't really care about absolute values because ultimately we just pick largest one and don't care what it was. So in training all other weights just get scaled by the bias so it can be 1. It's little like doing physics calculations with speed of light set to 1.
If you have simple feed forward network of perceptrons where ultimately in the end you just pick the largest output and don't care about absolute values then maybe you'd also be fine with just setting all perceptron bias terms to 1 and excluding them from the learning.
Is bias learnable in biological neurons? Doesn't activation potential threshold (or whatever it's called) rely on some chemistry and isn't the same for all neurons?
> Researchers had observed similar patterns in BERT, where "a surprisingly large amount of attention focuses on the delimiter token [SEP] and periods," which they argued was used by the model as a sort of no-op. The same summer at Meta, researchers studying vision transformers found similar behavior, observing that models would repurpose uninformative background patches as computational scratchpads. This seems to go…
And beyond that. Sometimes I feel like AI research is reinventing wheels that exist elsewhere. Maybe just the wheels, but still.
Earlier quoted context omitted.
I miss GANs. I understand that they are much harder to train than transformers for the same performance even with high data regime and high parameter regime, but there was such good optimization research and tricks that came out of them. The work on the capacity of discriminators was super cool.
> much harder to train than transformers There's plenty of GANs that use transformers. PWC seems to be redirecting to GitHub currently but IIRC about half of top scores on FFHQ256 were GANs with transformers in them. I know that the number 2 was, I saw it at CVPR. It was a lot smaller and had higher throughput than the diffusion models it was outperforming. Though the main reason diffusion took over was for the abili…
Earlier quoted context omitted.
> much harder to train than transformers There's plenty of GANs that use transformers. PWC seems to be redirecting to GitHub currently but IIRC about half of top scores on FFHQ256 were GANs with transformers in them. I know that the number 2 was, I saw it at CVPR. It was a lot smaller and had higher throughput than the diffusion models it was outperforming. Though the main reason diffusion took over was for the abili…
thanks! got any links if you can spare the time? i think the info on gans using transformers might be enough. wasn't aware!
[0] https://arxiv.org/abs/2211.05770
[1] https://arxiv.org/abs/2106.07631
[2] https://arxiv.org/abs/2206.02262
> Barbero et al. have shown that attention sinks serve as "pressure valves" preventing what researchers call "over-mixing"—a pathological state where deep models processing long sequences blur important distinctions between tokens. The presence of a sink draws attention away from other tokens, limiting the spread of information (and noise) and resulting in more stable embeddings. This sounds like it is working for th…
The point is that there's not always a right token to attend to. If the information you're looking for is not there, no clever attention scheme will find it. The best you can hope for when that happens is that the value returned in the "not found" case is distinguishable from the "found" case. Having an attention sink serve as a fixed "not found" value is one way to do this.
Earlier quoted context omitted.
The point is that there's not always a right token to attend to. If the information you're looking for is not there, no clever attention scheme will find it. The best you can hope for when that happens is that the value returned in the "not found" case is distinguishable from the "found" case. Having an attention sink serve as a fixed "not found" value is one way to do this.
Maybe another analogy (or at least the way I intuitively understood it) was that humans sometimes skip over tokens we know to be fluff/filler. Without a sink, models have no way to "skip" over a token, that token _will_ attend to all previous tokens and be incorporated into the residual stream. It's easy to see that for filler tokens this will tend to hurt quality more than improve it, since you're more likely to pul…
Earlier quoted context omitted.
thanks! got any links if you can spare the time? i think the info on gans using transformers might be enough. wasn't aware!
Sure. This was the paper[0]. Here's a few more you might find these interesting. Google's Transformer GAN[1] (not a transformer at all resolutions). Diffusion-GAN[2] is a hybrid architecture. Remember that technically the GAN process can use any underlying architecture. Arguably you could say some of the training steps in LLMs are GANs. And I think this one is also interesting in a similar respect[3]. Before PWC went…