Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

arrowtsx.dev

Post reply on HN
Print view View original

221–230 of 318 posts

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#221

Post by jmalicki »

jgalt212 wrote: post #218
jmalicki wrote: post #133

Earlier quoted context omitted.

I find these internet arguments talking about LLMs as if they are trained by reading the internet to be wild. Yes, pretraining still exists. But for the past few years, pretraining by reading the internet is just the initial bootstrapping of LLM training. The RL training they get from bespoke training data, with very very different characteristics than what these armchair analyses claim, dominates these days.

Outside of games and coding generating enough valid examples and counter-examples to harness the power of RL is cost prohibitive.

Which is why rubrics as rewards are used.

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#222

Post by ozgung »

utopiah wrote: post #205
wolttam wrote: post #113

> it is clear that actual intelligence has plateaued significantly. > Moving forward, the industry cannot continue to train bigger and bigger models since their intelligence not only plateaus but often will get worse These are wild claims - why are we concluding that bigger models and more data = more hallucination? That’s actually the opposite of what’s been happening over the last couple years. Some models may stil…

>> it is clear that actual intelligence has plateaued significantly. > These are wild claims - Indeed, it is not clear there was any actual intelligence at any point. A lot of generated content sure, sometimes even useful, but not necessarily anything more.

What is the definition of "actual intelligence"? How does it differ from regular intelligence and non-intelligence?

If someone can "design a custom asyncio event loop policy in that overrides get_child_watcher()", I would call that person intelligent. Does that mean that person is not actually intelligent but a mere content creation machine?

Traditionally if you can create content, this shows you're intelligent. Created content is often called "intellectual" property. If a person can understand complex ideas and make connection between them, that is considered intellectual work. You have to be intelligent to do intellectual work. If a person can solve problems, this is also called intelligence. If the person can solve more complex problems, that person is said to have higher intelligence. This is often measured with a scale called IQ (Intelligence Quotient). There are other types of intelligence but they are basically the variations of the same ability. Most definitions of intelligence also involve an ability to adapt into the environment.

Since intelligence is such a broad concept what exactly is the difference between the actual intelligence and AI, other than one is natural and the other one is artificial?

I understand being anti-AI because of the very real societal concerns. But ignoring what is in front of you is not a solution.

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#223

Post by sroussey »

MattRogish wrote: post #206
jmalicki wrote: post #186

Earlier quoted context omitted.

As a side gig, I write novel software that solves problems no existing software does, that existing LLMs have difficulty reproducing, purely for the purpose of existing as LLM training data. There are journalists being hired to write Atlantic-worthy articles that exist only as LLM training data, because they're getting paid more than the Atlantic would pay them for it. It's insane. Yes, they are hiring the experts th…

I'm not saying they are not trying - I'm saying we're inventing new problems faster than any Lab can: 1) Identify the gaps 2) Determine how to fix them 3) Implement a fix (especially if that fix is: identify and find experts) 4) And judge the result How do they know [person] is an expert in [some field]? How do they find that person? How many experts are necessary to give the right information? How do we evaluate the…

> How do they know [person] is an expert in [some field]? How do they find that person?

You just stumbled upon billion dollar businesses: Mercor, micro1, Scale AI, Surge AI, etc

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#224

Post by sroussey »

dominotw wrote: post #196
mcphage wrote: post #160

Earlier quoted context omitted.

Where do they get the bespoke training data from? And how much? I don’t really know anything about this.

meta has reallocated a significant protion of their staff to genrating this

Meta also reportedly took a 49% nonvoting stake in Scale AI in June 2025 for about $14.3–$14.8 billion.

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#225

Post by giardini »

jmalicki wrote: post #186
MattRogish wrote: post #181

Earlier quoted context omitted.

I'd have to imagine there are wildly diminishing marginal returns to additional SFT/post-training passes. There are a bounded number of (useful) derivations/combinations of Duff's device. If Frontier Labs wish to reduce hallucinations on factual things, they will have to hire people (or the data providers will need to) to do fundamental research above and beyond what is available in extant literature and the web. IE…

As a side gig, I write novel software that solves problems no existing software does, that existing LLMs have difficulty reproducing, purely for the purpose of existing as LLM training data. There are journalists being hired to write Atlantic-worthy articles that exist only as LLM training data, because they're getting paid more than the Atlantic would pay them for it. It's insane. Yes, they are hiring the experts th…

jmalicki says many things, among them being

"As a side gig, I write novel software that solves problems no existing software does,"

and

"Yes, they are hiring the experts themselves. To create new knowledge above and beyond what's on the internet. To be locked away as LLM training data."

More likely you're joking and/or paranoid!8-))

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#226

Post by giardini »

jmalicki wrote: post #210
MattRogish wrote: post #206

Earlier quoted context omitted.

I'm not saying they are not trying - I'm saying we're inventing new problems faster than any Lab can: 1) Identify the gaps 2) Determine how to fix them 3) Implement a fix (especially if that fix is: identify and find experts) 4) And judge the result How do they know [person] is an expert in [some field]? How do they find that person? How many experts are necessary to give the right information? How do we evaluate the…

> How do they know [person] is an expert in [some field]? How do they find that person? They have a PhD from a top school, they are a licensed attorney, they are a licensed physician, a board certified cardiologist, etc. They are constantly recruiting from these populations with well-paying side gigs. > 4) And judge the result That's what they pay the experts for. And to have experts review the other experts with pee…

Ahhhh! the ever-present omniscient "they" of paranoia!

But be careful: they are watching you and they don't want you giving away their secrets!

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#227

Post by bilater »

wolttam wrote: post #113

> it is clear that actual intelligence has plateaued significantly. > Moving forward, the industry cannot continue to train bigger and bigger models since their intelligence not only plateaus but often will get worse These are wild claims - why are we concluding that bigger models and more data = more hallucination? That’s actually the opposite of what’s been happening over the last couple years. Some models may stil…

Yeah not only is it totally unsubstantiated, the benchmarks are getting less useful to really show the difference between these models. Big model smell is still a thing and GLM 5.2 while impressive is not Fable class.

Here is something I would like people to chew on. Perhaps the smartest researchers in the world across multiple labs know more about this than we do? Perhaps they are aware of issues like the data wall and diminishing marginal returns. And perhaps they are being honest when they tell you there is no wall?

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#228

Post by jmalicki »

giardini wrote: post #225
jmalicki wrote: post #186

Earlier quoted context omitted.

As a side gig, I write novel software that solves problems no existing software does, that existing LLMs have difficulty reproducing, purely for the purpose of existing as LLM training data. There are journalists being hired to write Atlantic-worthy articles that exist only as LLM training data, because they're getting paid more than the Atlantic would pay them for it. It's insane. Yes, they are hiring the experts th…

jmalicki says many things, among them being "As a side gig, I write novel software that solves problems no existing software does," and "Yes, they are hiring the experts themselves. To create new knowledge above and beyond what's on the internet. To be locked away as LLM training data." More likely you're joking and/or paranoid!8-))

I wish our actual world wasn't an implausible scifi novel!

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#229

Post by stalfie »

roenxi wrote: post #155
stalfie wrote: post #147

Earlier quoted context omitted.

Well, I'd argue that this depends on the field you're investigating. Sometimes you have a way to identify objective reality and sometimes you don't. In mathematics the majority of the field is verifiable in this way. Coding a bit less as it's intersubjective, as and the ideal methodology is subject to taste. But even in muddy fields of reality like medicine, there are objective facts to be found. When someone comes i…

> In mathematics the majority of the field is verifiable in this way. Does mathematics count as not a hallucination though? Particularly in pure mathematics they take a certain pride coming up with wild concepts as unrooted as possible in anything relevant to human existence. The name of the game is purely about maintaining internal logical consistency - which is something an AI can do while hallucinating. AI halluci…

Mmmm, not sure I agree with this, although this is a topic where we would have to do a lot of groundwork to formulate our positions precisely in order to ensure we're actually discussing the same thing. My counterargument is that verified mathematics does exist. A lot of mathematical models of physics predicted the existence of stuff that experiments later verified, the higgs boson, antimatter and gravitational waves comes to mind. Terrence Tao did in fact make MR machines go faster simply by finding better maths, and the tumors those machines see can be cut out and touched.

Yes, there are mathematical concepts that seem to exist purely in the realm of mathematics, but maths often touches reality in a consistent way that reflect experimental results. This seems to imply that there is more to mathematics than just internal consistency. And the parts that do not correspond to any observation right now, might just reach out and touch reality in the future. It is possible to create logically consistent systems that have nothing to do with reality, but this is not the mathematics that most mathematicians are thinking about.

Observation is the final arbiter of fact. Maybe we don't have a general system to verify ALL facts, but many facts are 100% verifiable, although not most of them. "Beyond reasonable doubt" is of course the highest level of fact as far as the scientific method is concerned, but some facts are so far beyond reasonable doubt that you might as well just call them true. In the average living human body, there is a particular clump of tissues that consistently corresponds a concept most experts would describe as a "heart", and it does in fact pump blood. True fact.

Re: GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2

#230

Post by nomel »

giardini wrote: post #225
jmalicki wrote: post #186

Earlier quoted context omitted.

As a side gig, I write novel software that solves problems no existing software does, that existing LLMs have difficulty reproducing, purely for the purpose of existing as LLM training data. There are journalists being hired to write Atlantic-worthy articles that exist only as LLM training data, because they're getting paid more than the Atlantic would pay them for it. It's insane. Yes, they are hiring the experts th…

jmalicki says many things, among them being "As a side gig, I write novel software that solves problems no existing software does," and "Yes, they are hiring the experts themselves. To create new knowledge above and beyond what's on the internet. To be locked away as LLM training data." More likely you're joking and/or paranoid!8-))

> I write novel software that solves problems no existing software does

This is actually really easy to do if you step out of web/gui/crud and into something where you won't find public code, most ever, because it's trade secret. For example, manufacturing.

Post reply on HN