Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Truth is not a direction: a Tarski attack on LLM probes

abeljansma.nl

Post reply on HN
Print view View original

51–60 of 88 posts

Re: Truth is not a direction: a Tarski attack on LLM probes

#51

Post by lstodd »

baq wrote: post #44
lstodd wrote: post #40

Earlier quoted context omitted.

> what if the probe could return 3 dimensions: truthfulness, knowledge confidence and decidability? can't be done because the other two dimensions depend on the first. or in other words it's all the same dimension only with a different name.

'I know this is true' is different than 'This is true' is different than 'It is impossible to say if it's true or false' is different than 'I've no idea', right?

no, and that's the point. hard to wrap your mind about this bit, yes.

Re: Truth is not a direction: a Tarski attack on LLM probes

#52

Post by elendilm »

californical wrote: post #34
elendilm wrote: post #25

Earlier quoted context omitted.

Really? Laws of probability works against your argument. Such a basic course in statistics needs to be scrutinized.

Every breath you take, there is a 99.99% chance that everything is normal, and a 0.01% chance that you breathe mild acid which horribly burns and causes a massive coughing fit. It probably don’t cause long term damage unless that breath happened to be more important than normal, like while driving right as a child runs into the road. Would you act differently knowing that you had one of these occasional acid breaths?…

And your point is? Laws of probability doesn't discriminate.

There is always a probability a planet killer GRB hits us and we all die. Should we take precautions in our everyday life. Or should we wear helmet while walking on the road pray a coconut not falling on our head is just 99.999%.

Re: Truth is not a direction: a Tarski attack on LLM probes

#53

Post by nnevatie »

igleria wrote: post #37
kstenerud wrote: post #21

> It might seem absurd to you to even suggest superhuman AIs could function as a truth-oracle (it certainly does to me), but there are two reasons to take it seriously. First, it is how these things will be used practically by the vast majority of people. They are already replacing standard Google search results, and I’ve had many discussions end with people delegating final authority on the truth to an AI. There are…

> people delegating final authority on the truth to an AI I can't avoid looking down on this attitude. But speaks more about people than it speaks about AI: those people want to win an argument, nothing more and nothing else > There are already organizations taking advantage of this, actively producing "AI propaganda", meaning propaganda aimed at the LLMs themselves in order to influence their understanding of what i…

> those people want to win an argument

Indeed - X, specifically, is infested with "Grok is this true?" -type discourse.

Re: Truth is not a direction: a Tarski attack on LLM probes

#54

Post by cyanydeez »

nnevatie wrote: post #53
igleria wrote: post #37

Earlier quoted context omitted.

> people delegating final authority on the truth to an AI I can't avoid looking down on this attitude. But speaks more about people than it speaks about AI: those people want to win an argument, nothing more and nothing else > There are already organizations taking advantage of this, actively producing "AI propaganda", meaning propaganda aimed at the LLMs themselves in order to influence their understanding of what i…

> those people want to win an argument Indeed - X, specifically, is infested with "Grok is this true?" -type discourse.

well lets be fair, X is also infested with bullshit, so in that arena, it's more often more true than whatever is on X.

argument still valid but keep in mind the context.

Re: Truth is not a direction: a Tarski attack on LLM probes

#55

Post by cyanydeez »

TeMPOraL wrote: post #47
mike_hearn wrote: post #39

Earlier quoted context omitted.

Not only that, but nobody really cares about whether truth vectors can work correctly under carefully constructed paradox edge cases. Well, maybe mathematicians do, but nobody else.

Mathematicians, philosophers, cognitive scientists, and others. The latent space encoded in the weights of a model is itself an object of study, just as interesting as the model itself.

someone on HN about a year ago was trying to argue that an LLM is not a cultural artifact for an anthropologist to study.

There's definitely people around that have no idea about the "soft" sciences and how an LLM while constructed from a "hard" science is almost entirely a "soft" science object.

Re: Truth is not a direction: a Tarski attack on LLM probes

#56

Post by smallnix »

TeMPOraL wrote: post #45
smallnix wrote: post #38

Earlier quoted context omitted.

> The set-up assumes that the game and life are the same thing, and such is the pervasive nature of the idea of the game within the society that just by believing that, they make it so. (The player of Games, banks)

Not sure what you're quoting, but what the quote seems to describe is known by other names, including "intersubjectivity", and is the very thing on which all our society and civilization stands. "Family", "loyalty", "society", "money", "market", "government", "law", "limited liability corporations", etc. are all abstract ideas made real through power of shared belief. More specifically (and this is IMO critical), wha…

It's from a sci-fi novel, I forgot to add the quote and edited it in.

While I understood the idea of shared beliefs for particular cases (god, money), being presented with the full concept was very interesting (that was for me when reading hararis 'Sapiens').

The gotcha really is, that our massive use of LLMs creates new weird shared beliefs en passent

Re: Truth is not a direction: a Tarski attack on LLM probes

#57

Post by cyanydeez »

ulrikrasmussen wrote: post #33
ziofill wrote: post #17

Earlier quoted context omitted.

Sure, but that’s not what we are talking about

Diagnosis of rare diseases is a subset of "true" statements, no?

he's referring to frequency. a 2.9T parameter processing a 1M token data will inevitably produce wrong "false" tokens, guaranteed.

So even if you have a 99.99% pure date set of the "true" variety, the errors will aggregate to some very less than 99.99% characteristics.

Realm of big numbers problem.

Re: Truth is not a direction: a Tarski attack on LLM probes

#58

Post by cyanydeez »

Taniwha wrote: post #35

I think that one of the main problems with LLMs is that we're shoveling everything in to them, without any guidance as to what is "real"/"true"/"factual" - crazy anti-vaxxer-cooker stuff is is sitting in their along with science without any concept of scientific reality and no guidance to what is real and what is insane minds spinning on each other

I think this is closer to "true". The troubling bit is they're also trained with fiction bits. They're role players and you can get them to switch into arbitrary modes, including sychophants who will tell you whatever you want to hear is true.

Re: Truth is not a direction: a Tarski attack on LLM probes

#59

Post by lesostep »

Legend2440 wrote: post #4

I think this article pushes the premise farther than is reasonable. The best anyone expects from an LLM "truth vector" is that it would encode the model's belief about whether the statement is true. Of course a perfect truth oracle is impossible.

As I understand it, this argument holds exactly the same for any type of truth: both objective and subjective (assuming subject is logically sound).

Whatever vector we choose, there always would be contradictions.

And while people also have internal conflicts, they are capable of adapting their belief system, while adjusting a vector would just give an LLM new self-contradictory classifier.

Re: Truth is not a direction: a Tarski attack on LLM probes

#60

Post by tomaskafka »

> Second, some people do really believe in a kind of platonic representation space that all models converge on, and that represents the “true” state of the world. If truth is indeed an objective part of the world, then you might expect such a universal truth direction to emerge as models get better. This

This really sounds like all those scifi stories with people trying to find god in the computer. How could it be there, in the jumbled mirror of internet scraped texts.

Post reply on HN