Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Differential Transformer

arxiv.org

Post reply on HN
Print view View original

61–70 of 188 posts

Re: Differential Transformer

#61

Post by spywaregorilla »

spencerchubb wrote: post #27
Rhapso wrote: post #9

Earlier quoted context omitted.

"Hallucination" isn't really a problem that can be "fixed". Its just model error. The root problem is simply that the model doesn't capture reality, just an approximation. What we are incorrectly calling "hallucination" is just the best the model has to offer.

it's not "just" model error during pre-training, there is never an incentive for the model to say "I don't know" because it would be penalized. the model is incentivized to make an educated guess large transformer models are really good at approximating their dataset. there is no data on the internet about what LLMs know. and even if there were such data, it would probably become obsolete soon that being said, maybe…

> during pre-training, there is never an incentive for the model to say "I don't know" because it would be penalized. the model is incentivized to make an educated guess

The guess can be "I don't know". The base LLM would generally only say I don't know if it "knew" that it didn't know, which is not going to be very common. The tuned LLM would be the level responsible for trying to equate a lack of understanding to saying "I don't know"

Re: Differential Transformer

#62

Post by nowayno583 »

Does anyone understand why they are taking the difference between transformers instead of the sum? It seems to me that in a noise reducing solution we would be more interested in the sum, as random noise would cancel out and signal would be constructive.

Of course, even if I'm right proper training would account to that by inverting signs where appropriate. Still, it seems weird to present it as the difference, especially seeing as they compare this directly to noise cancelling headphones, where we sum both microphones inputs.

Re: Differential Transformer

#63

Post by dartos »

blackbear_ wrote: post #43
msoad wrote: post #7

Like most things in this new world of Machine Learning, I'm really confused why this works? The analogy to noise-cancelling headphones is helpful but in that case we clearly know which is signal and which is noise. Here, if we knew why would we even bother to the noise-cancelling work?

With a single softmax you cannot predict exactly 0, but only very small numbers. When you have a large number of values to add up, this "poisons" the output with a lot of irrelevant stuff (the noise mentioned in the paper). To make things worse, low attention values will have very low gradient, thus needing a lot of weight updates to undo that kind of mistakes. On the other hand, subtracting the output of two softmax…

> predict a weight of exactly zero for some of the values

Wouldn’t this be pretty unlikely, though?

Re: Differential Transformer

#64

Post by moffkalast »

dartos wrote: post #59

> By being less distracted by irrelevant context, Diff Transformer can mitigate hallucination in question answering and text summarization I’m very interested in this claim. I was under the impression that hallucination is unavoidable in these kinds of models. IIRC proof for that was trending on HN a couple weeks ago.

It's not possible to get rid of it entirely, but if you can get the model to bullshit only 0.1% of the time instead of 5% of the time it's a massive improvement.

Most of it should be happening when there's no data to draw conclusions from. E.g. STT models make up words in silence, vision models find things in lens cap noise, LLMs make up explanations when they have no data to pull from.

The real solution would be more along the lines of training models to specifically ignore these cases, or in the case of LLMs to just know when to say "I don't know".

Re: Differential Transformer

#65

Post by phire »

msoad wrote: post #7

Like most things in this new world of Machine Learning, I'm really confused why this works? The analogy to noise-cancelling headphones is helpful but in that case we clearly know which is signal and which is noise. Here, if we knew why would we even bother to the noise-cancelling work?

Noise cancelling headphones are probably the wrong analogy here.

The better example is the differential signalling used in professional audio and many digital signaling protocols like Ethernet, HDMI and USB.

Instead of using one wire, referencing to ground, they send the signal as the difference between both wires. Both wires end up carrying the same signal with inverted polarity. Because both wires are running next to each-other any external noice will be applied to both equally.

The voltage will change, but the difference in voltage between both wires is untouched. And when you subtract the two voltages at the receiver end, any noise simply gets subtracted out.

Re: Differential Transformer

#66

Post by singularity2001 »

spencerchubb wrote: post #27
Rhapso wrote: post #9

Earlier quoted context omitted.

"Hallucination" isn't really a problem that can be "fixed". Its just model error. The root problem is simply that the model doesn't capture reality, just an approximation. What we are incorrectly calling "hallucination" is just the best the model has to offer.

it's not "just" model error during pre-training, there is never an incentive for the model to say "I don't know" because it would be penalized. the model is incentivized to make an educated guess large transformer models are really good at approximating their dataset. there is no data on the internet about what LLMs know. and even if there were such data, it would probably become obsolete soon that being said, maybe…

in another paper which popped up recently they approximated uncertainty with Entropy and inserted "wait!" tokens whenever Entropy was high, simulating chain of thought within the system.

Re: Differential Transformer

#67

Post by ToValueFunfetti »

viraptor wrote: post #40
ToValueFunfetti wrote: post #36

Earlier quoted context omitted.

>Whether that sequence matches any kind of objective truth is orthogonal to how models work. Empirically, this cannot be true. If it were, it would be statistically shocking how often models coincidentally say true things. The training does not perfectly align the model with truth, but 'orthogonal' is off by a minimum of 45 degrees.

It matches the training data. Whether the training data matches truth (and whether it's correctly understood - sarcasm included) is a completely separate thing. > The training does not perfectly align the model with truth, but 'orthogonal' Nitpicky, but the more dimensions you have, the easier it is for almost everything to be orthogonal. ( https://softwaredoug.com/blog/2022/12/26/surpries-at-hi-dime... ) That's why…

I went to school to learn about the world and the overwhelming majority of that learning was from professors and textbooks. Whether the professors' beliefs and the textbooks' contents reflected the true properties of the world was a completely separate thing, entirely outside of my control. But I did come away with a better understanding of the world and few would say that education is orthogonal to that goal.

If you add two vectors that don't have a truth component (ie. are orthogonal to the truth), the resulting vector should be no closer to the truth. If you start with random weights and perform some operation on them such that the new weights have a higher likelihood of producing true statements, the operation must not have been orthogonal to the truth. Am I wrong there?

Re: Differential Transformer

#68

Post by ErikBjare »

dartos wrote: post #59

> By being less distracted by irrelevant context, Diff Transformer can mitigate hallucination in question answering and text summarization I’m very interested in this claim. I was under the impression that hallucination is unavoidable in these kinds of models. IIRC proof for that was trending on HN a couple weeks ago.

Mitigate, not completely fix.

Re: Differential Transformer

#69

Post by seamossfet »

msoad wrote: post #7

Like most things in this new world of Machine Learning, I'm really confused why this works? The analogy to noise-cancelling headphones is helpful but in that case we clearly know which is signal and which is noise. Here, if we knew why would we even bother to the noise-cancelling work?

It sounds like they're just splitting the query / key space down the middle. We don't know which dimensions are encoded in each matrix, but they're assuming the "noise" introduced in one query / key space is equivalent to noise introduced in the other space.

If that is the case, then the "signal" in this case would be the softmax that encodes the dimensions captured by the query / key space. Since the noise ideally is the same in both softmax encodings, subtracting them should "cancel out" the noise.

Re: Differential Transformer

#70

Post by dartos »

patcon wrote: post #34

I wonder what is lost here. Surely there's a trade-off... I'm wondering if there's any effect of "creativity", or ability to interpolate between concepts. Hallucination and creativity feel very related to me. I understand hallucinating as simply being misaligned with the space humans feel appropriate to interpolate between

> Hallucination and creativity feel very related to me.

Why? I see them as just sampling errors.

Sure a mistake can spark inspiration sometimes, but creativity is much more than mistakes.

> I understand hallucinating as simply being misaligned with the space humans feel appropriate to interpolate between

These language models are next-token predictors. The way the next token is predicted is by sampling a probability space outputted by the model.

That sampling process can be non deterministic.

Hallucinations are when that sampling results in tokens that come together to create a false or otherwise unintended statement.

You can just as well think of everything a model outputs as a hallucination, but we train the model to output a space what we want them to hallucinate is more likely. Otherwise it just outputs meaningless noise.

“Hallucinate” is really an awful word for what it’s trying to describe.

Post reply on HN