Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

arrowtsx.dev

Post reply on HN
Print view View original

101–110 of 318 posts

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#101

Post by skillina »

amelius wrote: post #66
stalfie wrote: post #60

One thing I wonder about hallucinations, is that it seems on the surface that it is an easy problem for RLVR to target. Since you're already generating enormous amounts of reasoning traces which are verified by correct answers, just have "don't know" as an option as a valid answer, and on problems where none of the thousands of reasoning traces led to a correct answer, just promote the traces that led to the "don't k…

But if an LLM says "I don't know" should you pay for the tokens?

"I don't know" has positive value, presumably you could prompt further to learn more about where it got stuck. It also increases the value of correct answers, by improving confidence that answers are actually correct.

"Confidently incorrect" has negative value. At best, a human realizes the answer is wrong and At worst, the incorrect information makes is not identified and can cause untold damage. By having the potential to be so severely wrong, it lessens the value of correct answers because there is a lower confidence value on their output.

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#102

Post by otabdeveloper4 »

embedding-shape wrote: post #91
macleginn wrote: post #83

Earlier quoted context omitted.

The main problem here is that hallucination suppression doesn’t generalise. We can penalise models for incorrect answers on a wide range of questions, but this doesn’t lead to the emergence of a coherent worldview, which, coupled with logical abilities, is the only true remedy against hallucinations. With current architectures, hallucinations will likely persist on open-domain tasks forever.

> We can penalise models for incorrect answers on a wide range of questions, but this doesn’t lead to the emergence of a coherent worldview, which, coupled with logical abilities, is the only true remedy against hallucinations I don't think anyone is trying to add "a coherent worldview" by reducing hallucinations, not sure how that even realistically could be aim. What people want, is for the models to stop giving co…

Model outputs don't have a confidence score.

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#103

Post by otabdeveloper4 »

realaleris149 wrote: post #45
gymbeaux wrote: post #17

Earlier quoted context omitted.

Those numbers are abysmal. Should we really be using LLMs to write our code? I have a theory- LLMs can spit out code that gets the job done and looks ok, maybe even great, but contains small “anomalies” that compound over time. An enterprise app developed entirely with LLM-happy devs might end up virtually unmaintainable. I’m not sure how to explain it, but the more I see LLM-written code the more I feel it’s bad cod…

Take a look at a sufficiently old random internal repo which was not written with LLMs and compare. My observation is that they are equally bad and hard to maintain or even more so than the new ones. One thing I’ve noticed is that the LLM assisted ones have a lot more comments which is nice but take more time to read.

Yes, LLMs generate technical debt.

And they do it faster than any human developer.

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#104

Post by ExtremisAndy »

nutjob2 wrote: post #98
wongarsu wrote: post #95

Earlier quoted context omitted.

Because nearly all benchmarks measure "accuracy" by giving you a point for a correct answer, and 0 points for everything else. If you have 100 questions you are 10% certain on, answering "I don't know" to all of those leads to 0 points, answering all of them as if you are confident leads to an expected value of 10 points. So that's what most AIs are trained to do AA-Omniscience is the only AI benchmark I know of wher…

It should be 1 for correct, 0 for don't know and -1 for wrong. They are much better incentives. In real life a wrong answer is much more damaging than a don't know.

See, this, to me, seems obvious, but I’m sure it’s more challenging/complex than I can imagine (I am NOT an expert on AI in any way imaginable). But there has to be a solution. Just yesterday I was asking Gemini to tell me about a certain college professor, and it gave me a list of facts about them. And it was perfect. Then, out of curiosity, I followed up with “tell me more about him!” and it spit out several more bits of information about this person that were entirely hallucinated (e.g., gave them credit for writing papers they didn’t write, said they won awards that actually someone else won). I know this is all complex and certainly beyond my limited skill set, but goodness, we’ve got to get this figured out with so many people depending on and trusting these things nowadays. It’s quite scary.

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#105

Post by derektank »

intended wrote: post #90
BDPW wrote: post #82

Earlier quoted context omitted.

If you had an llm that could accurately predict when a claim is uncertain it would be very popular, I think. I would pay for that kind of reliability tbh

This would break reality. There’s some underlying physical law that prevents the existence of any algorithm of truth.

If you accept certain axioms a priori, it’s fine. If you simply let the machine intelligence take it for granted that induction works because nature is uniform and give it some way to test its predictions, it would have all the building blocks it needs to reason out a lot of very useful information. Which as the parent comment points out, people would absolutely pay a lot of money for.

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#107

Post by vorticalbox »

otabdeveloper4 wrote: post #102
embedding-shape wrote: post #91

Earlier quoted context omitted.

> We can penalise models for incorrect answers on a wide range of questions, but this doesn’t lead to the emergence of a coherent worldview, which, coupled with logical abilities, is the only true remedy against hallucinations I don't think anyone is trying to add "a coherent worldview" by reducing hallucinations, not sure how that even realistically could be aim. What people want, is for the models to stop giving co…

Model outputs don't have a confidence score.

even if they did it it wouldn't be of much use because correct or not the output was the likely output 100% of the time.

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#108

Post by al_borland »

embedding-shape wrote: post #92
amelius wrote: post #66

Earlier quoted context omitted.

But if an LLM says "I don't know" should you pay for the tokens?

Depends on what your understanding of the product is. If someone sold you a "Solved all your problems" machine, and it suddenly doesn't solve all your problems, then probably no, you shouldn't pay. But the way I'm being sold LLMs, is basically "A text generator that gives your plausible-sounding human text that sometimes hallucinates and gets things wrong, based on your input", then regardless of what the outcome is,…

Let's pretend I am someone who has heard people talk about ChatGPT, but have on idea what it actually is. I go to the website and am not presented with any information, just a prompt. So I ask it what it is and what it can do for me.

My ask:

> In a couple sentences, explain to me the product I'm being sold with ChatGPT. What does it do for me?

The Reply from ChatGPT:

> ChatGPT is a conversational AI that helps you think, create, learn, analyze, and get things done faster. You can use it to answer questions, draft and edit writing, summarize information, brainstorm ideas, learn new topics, write code, plan projects, and increasingly act as an assistant that can search for information, work with documents, generate images, and help complete tasks.

> In simple terms: you're buying access to an AI that turns natural language into useful work—saving time, expanding your capabilities, and giving you an always-available collaborator for both everyday tasks and specialized knowledge work.

This sounds much more like the former, a "solve all your problems" machine.... not a plausible-sounding text generation machine.

Only two weeks ago Sam Altman said their new data center "could" be where cancer gets cured[0]. It is only the people who deeply understand AI who see it as a text generator of plausible-sounding text. That isn't what the marketing department, the CEO, or the product itself seem to be saying. I'm using OpenAI as the example here, but the others don't seem much different.

[0] https://www.youtube.com/watch?v=9-tOtbDDrJA

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#109

Post by realusername »

ben_w wrote: post #51
realusername wrote: post #19

Earlier quoted context omitted.

> code that gets the job done and looks ok, maybe even great, but contains small “anomalies” that compound over time They clearly are only assistants for the moment, you can use them to do work ... but only if you could do the said work yourself alone in the first place.

I would say "only if you can review said work yourself alone", rather than "do". I'm an experienced developer, but I don't count myself as a web dev or a python dev; I can review the web and python stuff I get out of the AI (sometimes I need to ask the AI follow-up questions so I can find official documentation for what it did), but I can't write it.

I think you could eventually do it then, it would just take you longer.
Post reply on HN