Truth is not a direction: a Tarski attack on LLM probes
1–10 of 88 posts
Re: Truth is not a direction: a Tarski attack on LLM probes
#2Fun, though as hinted at the end, the point of LLM "truth" probes is to measure the model's internal judgment of truthfulness. There's no reason this judgment, even if measured with 100% accuracy, couldn't be mistaken or logically inconsistent.
Re: Truth is not a direction: a Tarski attack on LLM probes
#3Someone at MIRI must know how to solve this. Good luck getting them to tell you how!
Re: Truth is not a direction: a Tarski attack on LLM probes
#4I think this article pushes the premise farther than is reasonable.
The best anyone expects from an LLM "truth vector" is that it would encode the model's belief about whether the statement is true. Of course a perfect truth oracle is impossible.
Re: Truth is not a direction: a Tarski attack on LLM probes
#5It looks like Zach Weinersmith predicted this exact line of research 11 years ago [1], when he suggested testing the liar sentence using fMRI.
The same analysis applies: the probe tells us what the LLM thinks about the truth value if the sentence, not the truth value of the sentence. I don't think anyone claimed that these probes were truth oracles.
Re: Truth is not a direction: a Tarski attack on LLM probes
#6Aren't there definitions of Truth that are not the negation of Falsehood? Can't there be a function True(x) that is not equal to !False(x)? Can't there be a third function Paradox(x) such that these counterexamples can be considered paradoxes and therefore outside of the truth?
I'm admittedly not a logician and don't formally study paradoxes, but I never quite understood the whole category of "this sentence is false" paradoxes, I feel that the sentence is paradox and therefore it's neither false nor true.
I do agree that it's a truth vector sounds like a silly panacea fantasy, though. But more logical formality is not the counter argument that would convince me of it, rather I believe that there's less formal and rigorous ways to get closer to truth.
Re: Truth is not a direction: a Tarski attack on LLM probes
#7I think this article pushes the premise farther than is reasonable. The best anyone expects from an LLM "truth vector" is that it would encode the model's belief about whether the statement is true. Of course a perfect truth oracle is impossible.
If anything, the whole vector space is the LLM's truth.
Re: Truth is not a direction: a Tarski attack on LLM probes
#8A direction that is 99.99% accurate survives this argument completely. For all practical purposes one does not need totality.
Re: Truth is not a direction: a Tarski attack on LLM probes
#9Aren't there definitions of Truth that are not the negation of Falsehood? Can't there be a function True(x) that is not equal to !False(x)? Can't there be a third function Paradox(x) such that these counterexamples can be considered paradoxes and therefore outside of the truth? I'm admittedly not a logician and don't formally study paradoxes, but I never quite understood the whole category of "this sentence is false"…
There are a LOT of different logical system.
Paraconsistent Logic
Intuitionistic Logic
Dialetheism
Re: Truth is not a direction: a Tarski attack on LLM probes
#10I think this article pushes the premise farther than is reasonable. The best anyone expects from an LLM "truth vector" is that it would encode the model's belief about whether the statement is true. Of course a perfect truth oracle is impossible.
If anything, the whole vector space is the LLM's truth.
LLMs are not optimized only for truth they are optimized for a more complicated objective that includes e.g. humans liking their output. It is a universal truth that to get humans to like you, you have to lie to them.