Ask HN: ChatGPT doesn't ackowledge being wrong?

For all the talk about "AI ethics" there is it seems strinking to me that the current state of the art model will opt for providing a convincing argument for why it's wrong answer is correct, rather than say that the answer it provides might be innacurate. (Funnily enough this is what humans tend to do aswell.) Being that these models often tend to be trained on data found on the internet, could this be a sign of the bias we tend to have when writing on social platforms and the internet in general? (As in justifying our answers instead of trying to get to the correct one)

So the questions are: 1. What are the consequences of this in the development of LLMs and in their application to various fields?

2. How would one implement this capability of recognizing where the model might be innacurate?

3. Is 2. really that much more complicated than what is currently being done? If not, then why hasn't it been done yet?

> AI: Answer that's clearly factually wrong (e.g. a function that doesn't exist or a completely wrong numerical figure).

> Me: that's not right, the function doesn't exist (etc.)

> AI: you are right, that function doesn't exist. The answer is blah

And repeat, since if it didn't get it right first time, it seems unlikely to be able to get there at all (and you'd not know when it did unless you already know the answer).

It's very like a specific person I know in a PR-marketing type job that will just glide across a noticed outright falsehood and instantly reshape what they're saying in real time and carry on as if nothing happened, leaving you wondering if you're taking crazy pills.

Impressive as it is, this kind of AI seems to be still working under what seems to me to be a possibly-flawed premise: training quantity has a sufficient quality of its own.

I can't prove that it's impossible with a clever enough system, but I simply don't see how you can get a right answer to come out of a statistical system that's been trained in input that might contain incorrect information, conflicting versions of it or just nothing at all, in which case it just makes something statistically plausible.

For example, it can give quite good answers about a well known event (e.g. a big earthquake), presumably because there are enough mentions of it in the training data. Ask about a footnote earthquake with few mentions and it will invent details that could be right, but aren't. For example a magnitude in the single digits "seems about right" and passes a sniff test, but has no factual basis in reality.

That said, I wonder if welding a large structured data store like Wolfram Alpha or Wikidata to the language model might resolve that issue: don't rely on statistics when the answer exists.

What's worse is that someone who isn't a domain expert might end up being convinced by the arguments and conviction the model provides.

Your second question is a good one, and I see that as a big problem with this generation of AI/ML. You're starting to scratch the surface of "this requires understanding of the real world" problems without models really understanding anything. It's all statistics and correlations in data, the model is not capable of really understanding its input or output.

These models are like very impressive Markov chain text generators in that they can spit out pretty convincing answers that seem cogent, but there is no real comprehension going on with what they read and write. It's just statistics.

Those things are NOT artificial intelligence.

They are specific noise generators. They generate noise that is as similar as possible to noise it learned from, which match the input.

THats it. nothing more.