Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

arrowtsx.dev

Post reply on HN
Print view View original

131–140 of 318 posts

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#131

Post by maxbond »

stalfie wrote: post #60

One thing I wonder about hallucinations, is that it seems on the surface that it is an easy problem for RLVR to target. Since you're already generating enormous amounts of reasoning traces which are verified by correct answers, just have "don't know" as an option as a valid answer, and on problems where none of the thousands of reasoning traces led to a correct answer, just promote the traces that led to the "don't k…

If you could write that reward function you wouldn't need an LLM, you'd just query the reward function to answer any question. You can create a benchmark and check that automatically, but you can't solve this in the general case. The model can do well on the benchmark but still give overconfident answers in areas the benchmark doesn't cover.

You can definitely tune a model to say "I don't know" more often but it will cost you performance, the model will reject some questions that it could answer meaningfully. In the degenerate case the model could collapse predicting that sequence always or almost always.

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#132

Post by maxbond »

nutjob2 wrote: post #98
wongarsu wrote: post #95

Earlier quoted context omitted.

Because nearly all benchmarks measure "accuracy" by giving you a point for a correct answer, and 0 points for everything else. If you have 100 questions you are 10% certain on, answering "I don't know" to all of those leads to 0 points, answering all of them as if you are confident leads to an expected value of 10 points. So that's what most AIs are trained to do AA-Omniscience is the only AI benchmark I know of wher…

It should be 1 for correct, 0 for don't know and -1 for wrong. They are much better incentives. In real life a wrong answer is much more damaging than a don't know.

And also because it creates "one neat trick" where it can answer "I don't know" for many/most things and still get credit.

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#133

Post by jmalicki »

an0malous wrote: post #121
wolttam wrote: post #113

> it is clear that actual intelligence has plateaued significantly. > Moving forward, the industry cannot continue to train bigger and bigger models since their intelligence not only plateaus but often will get worse These are wild claims - why are we concluding that bigger models and more data = more hallucination? That’s actually the opposite of what’s been happening over the last couple years. Some models may stil…

> why are we concluding that bigger models and more data = more hallucination? That’s not what your quotes said. They said bigger models = plateau in intelligence, nothing about more data or increased hallucinations The relevant quote for what you’re talking about would be: > It’s been proven that when a model is trained on large volumes of highly factual and non-theoretical data, it learns to always have an answer.…

I find these internet arguments talking about LLMs as if they are trained by reading the internet to be wild.

Yes, pretraining still exists. But for the past few years, pretraining by reading the internet is just the initial bootstrapping of LLM training. The RL training they get from bespoke training data, with very very different characteristics than what these armchair analyses claim, dominates these days.

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#134

Post by baq »

SeriousM wrote: post #94
solid_fuel wrote: post #3

> It’s been proven that when a model is trained on large volumes of highly factual and non-theoretical data, it learns to always have an answer. DeepSeek V4 Pro (1.6T params, 49B active, 44 AA Intelligence Index score) has a ludicrous 94% hallucination score on the AA-Omniscience benchmark, meaning on questions that it couldn’t figure out, it only stated that it didn’t know around 6% of the time, and the rest it conf…

There is no concept of "knowledge" in LLM as it is on Wikipedia. The question-tokens define the answer-tokens. That's it. The art relies in clustering the relevant weights together.

If it were that simple we’d all be talking with sql and yet this isn’t happening.

Circuits which emerge in the layers during training are much more complicated than a simple Bayesian relation.

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#135

Post by maxbond »

amelius wrote: post #66
stalfie wrote: post #60

One thing I wonder about hallucinations, is that it seems on the surface that it is an easy problem for RLVR to target. Since you're already generating enormous amounts of reasoning traces which are verified by correct answers, just have "don't know" as an option as a valid answer, and on problems where none of the thousands of reasoning traces led to a correct answer, just promote the traces that led to the "don't k…

But if an LLM says "I don't know" should you pay for the tokens?

Would you rather pay for a nonsensical explanation?

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#136

Post by andai »

> GPT-5.5 and DeepSeek V4 Pro are two of the clearest hallucination leaders, despite being absolutely huge. Because of their immense size they simply did not learn how to say “I don’t know” or recognize intricate logical and technical fallacies.

This implies that bigger models are more likely to hallucinate? That doesn't match my experience.

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#138

Post by verdverm »

EbNar wrote: post #79

The fact that a huge amount uf parameters may lead to worse hallucinations is something I didn't think of. Would this somewhat imply that DeepSeek V4 flash should be less prone to these issues?

small models cannot encode so many facts, they will hallucinate more out-of-box

a key method to help with hallucinations is to provide good sources when asking questions (context engineering / knowledge base)

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#139

Post by __natty__ »

hit8run wrote: post #111
remix2000 wrote: post #110

Calling llm slop "hallucinating" is so counter-productive imo. After all, LLMs are just a variant of markov chains and as such this technology isn't able to discern falsehoods from truths. It's like trying to use a barometer to tell the time.

You are also just a variant of markov chains wired in your brain. So what you complaining about?

And often it’s not perfect either. Just because one is true it doesn’t dismiss the other

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#140

Post by stalfie »

maxbond wrote: post #131
stalfie wrote: post #60

One thing I wonder about hallucinations, is that it seems on the surface that it is an easy problem for RLVR to target. Since you're already generating enormous amounts of reasoning traces which are verified by correct answers, just have "don't know" as an option as a valid answer, and on problems where none of the thousands of reasoning traces led to a correct answer, just promote the traces that led to the "don't k…

If you could write that reward function you wouldn't need an LLM, you'd just query the reward function to answer any question. You can create a benchmark and check that automatically, but you can't solve this in the general case. The model can do well on the benchmark but still give overconfident answers in areas the benchmark doesn't cover. You can definitely tune a model to say "I don't know" more often but it will…

I guess so. Just to be clear, I was talking about post-training methods for reasoning models here, not pre-training. I think "model as a judge" should actually do okay as a "sentiment analysis" style reward for expressing uncertainty. So if none of the thousands of reasoning traces you generate reach the validated answer, you run a judge to rate uncertainty and put those reasoning traces back into the training pool.

But I guess my logic breaks down here a bit, because if there is such a thing as a validated answer, then the correct answer is in fact never uncertainty. The correct answer is to continue post training until the model gets it right. So perhaps the real answer is to create RLVR tasks where the valid answer is "I don't know" and nothing else, like this benchmark does. Or maybe that doesn't work either, no matter how many you create.

I feel as though there is some kind of philosophical lesson to be had from how hard hallucinations are to get rid of. Maybe, similarly to humans, successful models are often "arrogant" in a sense. Perhaps you just never solve an Erdös problem without some degree of self deception that it's possible for you to do so. In this line of thinking, greatness in humans is actually not related to humility, but just being so good that you actually get things right when you try. Expressing humility is of course something great people tend to do, but I'm referring to what happens under the hood.

If you squint a bit, that's kinda the trend with models. The useful ones are not that much less likely to hallucinate, they are just good enough that they tend to get it right. This comparison is of course probably not even remotely correct, but at least it's fun to anthropomorphize a bit.

Post reply on HN