Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Truth is not a direction: a Tarski attack on LLM probes

abeljansma.nl

Post reply on HN
Print view View original

21–30 of 88 posts

Re: Truth is not a direction: a Tarski attack on LLM probes

#21

Post by kstenerud »

> It might seem absurd to you to even suggest superhuman AIs could function as a truth-oracle (it certainly does to me), but there are two reasons to take it seriously. First, it is how these things will be used practically by the vast majority of people. They are already replacing standard Google search results, and I’ve had many discussions end with people delegating final authority on the truth to an AI.

There are already organizations taking advantage of this, actively producing "AI propaganda", meaning propaganda aimed at the LLMs themselves in order to influence their understanding of what is truthful and bend it towards powerful actors' agendas. They're not even hiding the fact that they're doing this.

Re: Truth is not a direction: a Tarski attack on LLM probes

#22

Post by aesthesia »

galaxyLogic wrote: post #11
TZubiri wrote: post #6

Aren't there definitions of Truth that are not the negation of Falsehood? Can't there be a function True(x) that is not equal to !False(x)? Can't there be a third function Paradox(x) such that these counterexamples can be considered paradoxes and therefore outside of the truth? I'm admittedly not a logician and don't formally study paradoxes, but I never quite understood the whole category of "this sentence is false"…

Interesting point about "self-referential sentences". I tend to agree. In my view a sentence saying something like "This sentence ..." does not have valid semantic meaning. It says nothing because, what "This" in "This sentence" means is ill-defined. If terms we use are not well-defined, then sentences using such terms can not have meaning. But for the sake of argument let's explore, what could the "this" in (so call…

Quines produce similar issues to self-referential sentences without being directly self-referential. e.g. "'Yields falsehood when preceded by its quotation' yields falsehood when preceded by its quotation."

Re: Truth is not a direction: a Tarski attack on LLM probes

#23

Post by TZubiri »

erehweb wrote: post #20
galaxyLogic wrote: post #11

Earlier quoted context omitted.

Interesting point about "self-referential sentences". I tend to agree. In my view a sentence saying something like "This sentence ..." does not have valid semantic meaning. It says nothing because, what "This" in "This sentence" means is ill-defined. If terms we use are not well-defined, then sentences using such terms can not have meaning. But for the sake of argument let's explore, what could the "this" in (so call…

As the article notes, the sentence "This sentence is written in English" is well understood, and true. "This sentence is written in French" is also well understood and false.

Right, my original comment wasn't about self-reference, but paradoxes, and not all self-references result in paradoxes.

Not sure what the difference between the language of a sentence and the truthiness of a sentence is. Maybe it has to do with the fact that the language refers to syntactical features that can be determined at 'compile' time, while truthiness refers to a semantic quality feature that can only be determined at 'runtime'. (See en.wikipedia.org/wiki/colorless_green_ideas_sleep_furiously for at the very least a funny canonical example of a syntactically valid, but semantically nonsensical sentence.)

The fact that the language function can also be mapped to reduced parts of the sentence, might also be relevant, you can deduce that "This-" and "This sentence is-" are English, while you cannot partially compute that "This sentence is-" is true. A single negation bit being enough to change the result of the operation.

Re: Truth is not a direction: a Tarski attack on LLM probes

#24

Post by elendilm »

zarzavat wrote: post #10
TZubiri wrote: post #7

Earlier quoted context omitted.

If anything, the whole vector space is the LLM's truth.

LLMs are not optimized only for truth they are optimized for a more complicated objective that includes e.g. humans liking their output. It is a universal truth that to get humans to like you, you have to lie to them.

You don't represent all of us buddy. Some of us love truth above all else.

Re: Truth is not a direction: a Tarski attack on LLM probes

#25

Post by elendilm »

voxl wrote: post #16
ziofill wrote: post #8

A direction that is 99.99% accurate survives this argument completely. For all practical purposes one does not need totality.

A basic course in statistics will inform you of why a 99.99% accurate test should be looked at with skepticism when diagnosing a rare disease. Yet we see the fancy 9s and think somehow this many 9s is enough.

Really? Laws of probability works against your argument.

Such a basic course in statistics needs to be scrutinized.

Re: Truth is not a direction: a Tarski attack on LLM probes

#26

Post by TZubiri »

aesthesia wrote: post #22
galaxyLogic wrote: post #11

Earlier quoted context omitted.

Interesting point about "self-referential sentences". I tend to agree. In my view a sentence saying something like "This sentence ..." does not have valid semantic meaning. It says nothing because, what "This" in "This sentence" means is ill-defined. If terms we use are not well-defined, then sentences using such terms can not have meaning. But for the sake of argument let's explore, what could the "this" in (so call…

Quines produce similar issues to self-referential sentences without being directly self-referential. e.g. "'Yields falsehood when preceded by its quotation' yields falsehood when preceded by its quotation."

> "'Yields falsehood when preceded by its quotation' yields falsehood when preceded by its quotation."

Here be the limits of my brain.

It does feel (vibes) like an obfuscation of the self-referential canonical 'This sentence is false' example, like a sum of obfuscation techniques that are designed to confuse, but don't materially change the nature of the phenomenon:

1- reference is made implicit, instead of explicit

2- References another object, necessitating (at least) two instances of the same sentence, with one referencing the other.

3- Maybe some unnecessarily complex language that could be made simpler?

So what I'm getting out of it is that it feels like a mental trap that is hard to compute and understand because it was designed that way (or because it evolved that way), not necessarily because of it containing a fundamentally useful knowledge, it's difficulty to parse IS itself the interesting property of the sentence.

Maybe meta analysis like looking into the history of this sentence and seeing how many people went crazy going down that rabbit hole would be more enlightening than engaging in it in good faith. Which might only be useful if you have a high enough IQ that it doesn't confuse you any longer.

Re: Truth is not a direction: a Tarski attack on LLM probes

#27

Post by ngruhn »

elendilm wrote: post #24
zarzavat wrote: post #10

Earlier quoted context omitted.

LLMs are not optimized only for truth they are optimized for a more complicated objective that includes e.g. humans liking their output. It is a universal truth that to get humans to like you, you have to lie to them.

You don't represent all of us buddy. Some of us love truth above all else.

Not many though

Re: Truth is not a direction: a Tarski attack on LLM probes

#28

Post by baq »

Title is a bit clickbaitish, but the content is well worth reading - came in with my pitchfork ready and left agreeing with basically all of it, with questions like ‘what if the probe could return 3 dimensions: truthfulness, knowledge confidence and decidability?’

Also the observation that people treat LLMs like oracles when they’re everything but is spot on, something I’ve also been thinking about and it’s quite a bit scary.

Re: Truth is not a direction: a Tarski attack on LLM probes

#30

Post by samlinnfer »

Legend2440 wrote: post #4

I think this article pushes the premise farther than is reasonable. The best anyone expects from an LLM "truth vector" is that it would encode the model's belief about whether the statement is true. Of course a perfect truth oracle is impossible.

Ah but is the model's belief of the statements 1. complete, 2. consistent, 3. decidable?
Post reply on HN