Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Truth is not a direction: a Tarski attack on LLM probes

abeljansma.nl

Post reply on HN
Print view View original

71–80 of 88 posts

Re: Truth is not a direction: a Tarski attack on LLM probes

#71

Post by lelanthran »

elendilm wrote: post #24
zarzavat wrote: post #10

Earlier quoted context omitted.

LLMs are not optimized only for truth they are optimized for a more complicated objective that includes e.g. humans liking their output. It is a universal truth that to get humans to like you, you have to lie to them.

You don't represent all of us buddy. Some of us love truth above all else.

> You don't represent all of us buddy. Some of us love truth above all else.

So? Some people get off on pain, doesn't make the statement "People try to avoid pain" false.

Re: Truth is not a direction: a Tarski attack on LLM probes

#72

Post by lstodd »

baq wrote: post #66
lstodd wrote: post #51

Earlier quoted context omitted.

no, and that's the point. hard to wrap your mind about this bit, yes.

from a practical point of view the point doesn't have any value, assuming it's even true; it'll take some convincing

I'd say it's equivalent to holding that any one worldview has no value in itself. Which is strictly true, but also conveys zero information.

Re: Truth is not a direction: a Tarski attack on LLM probes

#73

Post by ziofill »

ziofill wrote: post #8

A direction that is 99.99% accurate survives this argument completely. For all practical purposes one does not need totality.

Ironically, some users took the “all” in my first post too literally. It should have read “for most practical purposes”

Re: Truth is not a direction: a Tarski attack on LLM probes

#74

Post by ziofill »

californical wrote: post #34
elendilm wrote: post #25

Earlier quoted context omitted.

Really? Laws of probability works against your argument. Such a basic course in statistics needs to be scrutinized.

Every breath you take, there is a 99.99% chance that everything is normal, and a 0.01% chance that you breathe mild acid which horribly burns and causes a massive coughing fit. It probably don’t cause long term damage unless that breath happened to be more important than normal, like while driving right as a child runs into the road. Would you act differently knowing that you had one of these occasional acid breaths?…

You are correct, but this is a made up example, not a practical matter. You’re falling into the same trap as the author.

Re: Truth is not a direction: a Tarski attack on LLM probes

#75

Post by anamax »

kstenerud wrote: post #21

> It might seem absurd to you to even suggest superhuman AIs could function as a truth-oracle (it certainly does to me), but there are two reasons to take it seriously. First, it is how these things will be used practically by the vast majority of people. They are already replacing standard Google search results, and I’ve had many discussions end with people delegating final authority on the truth to an AI. There are…

> There are already organizations taking advantage of this, actively producing "AI propaganda", meaning propaganda aimed at the LLMs themselves in order to influence their understanding of what is truthful and bend it towards powerful actors' agendas. They're not even hiding the fact that they're doing this.

Very similar techniques were used to influence web search.

And before that, 1984.

And it wasn't new then.

Re: Truth is not a direction: a Tarski attack on LLM probes

#76

Post by scotty79 »

layer8 wrote: post #69
scotty79 wrote: post #29

> this sentence has no proof As a programmer I was never impressed in such paradoxes. For me it was kinda obvious that in any sufficiently complex language you can create eqivalent of buggy infinite loop/recursion.

The issue is that you can’t generally determine whether a statement is “buggy” in that way, because under the assumption that you could, you can construct another paradox. Regarding programming languages, it means that for any programming language with a non-trivial type system, either there will be correctly typed programs that however fails the type checker, or there will be ill-typed programs that cause the type c…

> The issue is that you can’t generally determine whether a statement is “buggy” in that way, because under the assumption that you could, you can construct another paradox.

You can't always determine if a program is going to finish either because that would require infinite number of operations is some cases (so it would be a program that never finishes as well). So the analogy still stands.

Re: Truth is not a direction: a Tarski attack on LLM probes

#77

Post by Legend2440 »

samlinnfer wrote: post #30
Legend2440 wrote: post #4

I think this article pushes the premise farther than is reasonable. The best anyone expects from an LLM "truth vector" is that it would encode the model's belief about whether the statement is true. Of course a perfect truth oracle is impossible.

Ah but is the model's belief of the statements 1. complete, 2. consistent, 3. decidable?

I think this is a category error. Those are properties of systems of logical axioms.

But LLMs are not logical; they are statistical. It can only 'believe' a statement is true in the Bayesian sense, where the statement agrees with the priors.

Re: Truth is not a direction: a Tarski attack on LLM probes

#78

Post by HappMacDonald »

Ukv wrote: post #63
customguy wrote: post #31

Earlier quoted context omitted.

For a one time coin flip, sure. For a lot of them, depending on the stakes, that can be very, very wrong. https://www.google.com/search?q=why+99.99+accuracy+is+not+en... It doesn't take a lot of thought experiments to realize this. Imagine if every bite of food we eat had a 0.01% chance to turn into something instantly lethal in our mouth. Average lifespans would be reduced measurably, and apart from anxiety, we'd de…

ziofill's claim was that "A direction [in an LLM's embedding vector space] that is 99.99% accurate" is fine for practical purposes, not that 99.99% is fine for the chance of any given bite of food not killing you or similar hypotheticals - you'd want a few more 9s there. To justify relevance of inability to correctly answer liars-paradox-type questions ("what won't your response to this be?"), the article suggested t…

I think you are making a lot of assumptions on what "practical purposes" even mean in that case.

Please think carefully and then try to tell me whether or not some Trump-administration government agency would not in the "99.99% reliable" circumstance just plug an LLM into the nukes and funnel worldstate input into it and have it make the decision "is it time to fire the nukes?" over and over again each second.

I argue that that would require far more nines than even "will food turn to poison in my mouth" would.

Re: Truth is not a direction: a Tarski attack on LLM probes

#79

Post by HappMacDonald »

layer8 wrote: post #70
TZubiri wrote: post #6

Aren't there definitions of Truth that are not the negation of Falsehood? Can't there be a function True(x) that is not equal to !False(x)? Can't there be a third function Paradox(x) such that these counterexamples can be considered paradoxes and therefore outside of the truth? I'm admittedly not a logician and don't formally study paradoxes, but I never quite understood the whole category of "this sentence is false"…

You run into the same problems when you have True(x) = NOT (False(x) OR Paradox(x)). You can define NotTrue(x) := False(x) OR Paradox(x), and then you have True(x) = NOT NotTrue(x), which one can build the usual diagonalizations on.

Well, I mean True(x) does = NOT (NOT True(x)), so..

Re: Truth is not a direction: a Tarski attack on LLM probes

#80

Post by layer8 »

scotty79 wrote: post #76
layer8 wrote: post #69

Earlier quoted context omitted.

The issue is that you can’t generally determine whether a statement is “buggy” in that way, because under the assumption that you could, you can construct another paradox. Regarding programming languages, it means that for any programming language with a non-trivial type system, either there will be correctly typed programs that however fails the type checker, or there will be ill-typed programs that cause the type c…

> The issue is that you can’t generally determine whether a statement is “buggy” in that way, because under the assumption that you could, you can construct another paradox. You can't always determine if a program is going to finish either because that would require infinite number of operations is some cases (so it would be a program that never finishes as well). So the analogy still stands.

The reason you can't always determine if a program is going to finish is due to the undecidability of the halting problem, which is equivalent to Gödel’s first incompleteness theorem (you can prove one from the other). But that's not merely because programs might run into an infinite loop or recursion. It's deeper than that.
Post reply on HN