Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Differential Transformer

arxiv.org

Post reply on HN
Print view View original

31–40 of 188 posts

Re: Differential Transformer

#31

Post by CooCooCaCha »

hoosieree wrote: post #23
dilap wrote: post #15

Earlier quoted context omitted.

it can be fixed in theory if the model knows-what-it-knows, to avoid saying things its uncertain about (this is what (some) humans do to reduce the frequency w which they say untrue things). theres some promising research using this idea, tho i dont have it at hand.

LLMs can't hallucinate. They generate the next most likely token in a sequence. Whether that sequence matches any kind of objective truth is orthogonal to how models work. I suppose depending on your point of view, LLMs either can't hallucinate, or that's all they can do .

Whenever someone takes issue with using the word “hallucinate” with LLMs I get the impression they’re trying to convince me that hallucination is good.

Why do you care so much about this particular issue? And why can’t hallucination be something we can aim to improve?

Re: Differential Transformer

#32

Post by ithkuil »

AnimalMuppet wrote: post #17
dilap wrote: post #15

Earlier quoted context omitted.

it can be fixed in theory if the model knows-what-it-knows, to avoid saying things its uncertain about (this is what (some) humans do to reduce the frequency w which they say untrue things). theres some promising research using this idea, tho i dont have it at hand.

I'm pretty sure there's something I don't understand, but: Doesn't an LLM pick the "most probable next symbol" (or, depending on temperature, one of the most probable next symbols)? To do that, doesn't it have to have some idea of what the probability is? Couldn't it then, if the probability falls below some threshold, say "I don't know" instead of giving what it knows is a low-probability answer?

This may work when the next token is a key concept but when it's a filler word or a part of one of many sequences of words that can convey the same meaning but in different ways (synonyms but not only at the word also at the sentence levels) then it's harder to know whether the probability is low because the word is absolutely unlikely or because it's likelihood is spread/shared among other truthful statements

Re: Differential Transformer

#33

Post by digdugdirk »

Is there any way to replicate this with existing models, or are we going to need to wait for models to be trained in this style?

I'm imagining a smaller model examining the output tokens of a larger model and metaphorically slapping it on the wrist with a ruler if the output tokens start drifting off topic. Not quite the same, but an entertaining thought nonetheless.

Re: Differential Transformer

#34

Post by patcon »

I wonder what is lost here. Surely there's a trade-off...

I'm wondering if there's any effect of "creativity", or ability to interpolate between concepts. Hallucination and creativity feel very related to me. I understand hallucinating as simply being misaligned with the space humans feel appropriate to interpolate between

Re: Differential Transformer

#35

Post by causal »

digdugdirk wrote: post #33

Is there any way to replicate this with existing models, or are we going to need to wait for models to be trained in this style? I'm imagining a smaller model examining the output tokens of a larger model and metaphorically slapping it on the wrist with a ruler if the output tokens start drifting off topic. Not quite the same, but an entertaining thought nonetheless.

It's a different attention mechanism with a different map setup, so fundamentally a different type of model

Re: Differential Transformer

#36

Post by ToValueFunfetti »

hoosieree wrote: post #23
dilap wrote: post #15

Earlier quoted context omitted.

it can be fixed in theory if the model knows-what-it-knows, to avoid saying things its uncertain about (this is what (some) humans do to reduce the frequency w which they say untrue things). theres some promising research using this idea, tho i dont have it at hand.

LLMs can't hallucinate. They generate the next most likely token in a sequence. Whether that sequence matches any kind of objective truth is orthogonal to how models work. I suppose depending on your point of view, LLMs either can't hallucinate, or that's all they can do .

>Whether that sequence matches any kind of objective truth is orthogonal to how models work.

Empirically, this cannot be true. If it were, it would be statistically shocking how often models coincidentally say true things. The training does not perfectly align the model with truth, but 'orthogonal' is off by a minimum of 45 degrees.

Re: Differential Transformer

#37

Post by happypumpkin »

spencerchubb wrote: post #27
Rhapso wrote: post #9

Earlier quoted context omitted.

"Hallucination" isn't really a problem that can be "fixed". Its just model error. The root problem is simply that the model doesn't capture reality, just an approximation. What we are incorrectly calling "hallucination" is just the best the model has to offer.

it's not "just" model error during pre-training, there is never an incentive for the model to say "I don't know" because it would be penalized. the model is incentivized to make an educated guess large transformer models are really good at approximating their dataset. there is no data on the internet about what LLMs know. and even if there were such data, it would probably become obsolete soon that being said, maybe…

> it would probably become obsolete soon

Suppose there are many times more posts about something one generation of LLMs can't do (arithmetic, tic-tac-toe, whatever), than posts about how the next generation of models can do that task successfully. I think this is probably the case.

While I doubt it will happen, it would be somewhat funny if training on that text caused a future model to claim it can't do something that it "should" be able to because it internalized that it was an LLM and "LLMs can't do X."

Re: Differential Transformer

#38

Post by visarga »

dilap wrote: post #15
Rhapso wrote: post #9

Earlier quoted context omitted.

"Hallucination" isn't really a problem that can be "fixed". Its just model error. The root problem is simply that the model doesn't capture reality, just an approximation. What we are incorrectly calling "hallucination" is just the best the model has to offer.

it can be fixed in theory if the model knows-what-it-knows, to avoid saying things its uncertain about (this is what (some) humans do to reduce the frequency w which they say untrue things). theres some promising research using this idea, tho i dont have it at hand.

This reminds me it's easy to train similarity models, hard to train identity/equivalence prediction. Two strings can be similar in many ways, like "Address Line 1" and "Address Line 2" or "Position_X" and "Position_Y", yet distinct in meaning. That one character makes all the difference. On the other hand "Vendor Name" is equivalent with "Seller Company" even though they are pretty different lexically.

The dot product, which is at the core of attention, is good for similarity not identity. I think this is why models hallucinate - how can they tell the distinction between "I have trained on this fact" and "Looks like something I trained on".

Re: Differential Transformer

#39

Post by watsonmusic »

patcon wrote: post #34

I wonder what is lost here. Surely there's a trade-off... I'm wondering if there's any effect of "creativity", or ability to interpolate between concepts. Hallucination and creativity feel very related to me. I understand hallucinating as simply being misaligned with the space humans feel appropriate to interpolate between

not all hallucinations are creativity Imaginate that for a RAG application, the model is supposed to follow the given documents

Re: Differential Transformer

#40

Post by viraptor »

ToValueFunfetti wrote: post #36
hoosieree wrote: post #23

Earlier quoted context omitted.

LLMs can't hallucinate. They generate the next most likely token in a sequence. Whether that sequence matches any kind of objective truth is orthogonal to how models work. I suppose depending on your point of view, LLMs either can't hallucinate, or that's all they can do .

>Whether that sequence matches any kind of objective truth is orthogonal to how models work. Empirically, this cannot be true. If it were, it would be statistically shocking how often models coincidentally say true things. The training does not perfectly align the model with truth, but 'orthogonal' is off by a minimum of 45 degrees.

It matches the training data. Whether the training data matches truth (and whether it's correctly understood - sarcasm included) is a completely separate thing.

> The training does not perfectly align the model with truth, but 'orthogonal'

Nitpicky, but the more dimensions you have, the easier it is for almost everything to be orthogonal. (https://softwaredoug.com/blog/2022/12/26/surpries-at-hi-dime...) That's why averaging embeddings works.

Post reply on HN