Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Making o1, o3, and Sonnet 3.7 hallucinate for everyone

bengarcia.dev

Post reply on HN
Print view View original

51–60 of 233 posts

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#51

Post by miningape »

latexr wrote: post #43

> Conclusion > LLMs are really smart most of the time. No, the conclusion is they’re never “smart”. All they do is regurgitate text which resembles a continuation of what came before, and sometimes—but with zero guarantees—that text aligns with reality.

This, thank you. It pisses me off to no end when people pretend LLMs are smart. They are nothing but a well trained random text generator.

Seriously, some these conversations feel like interacting someone who believes casting bones and astrology are accurate. Likely because in both cases they are a result of confirmation bias.

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#52

Post by miningape »

wrs wrote: post #42
adamgordonbell wrote: post #37

We at pulumi started treating some hallucinations like this as feature requests. Sometimes an llm will hallucination a flag, or option that really makes sense - it just doesn't actually exist.

This sort of hallucination happens to me frequently with AWS infrastructure questions. Which is depressing because I can't do anything but agree, "yeah, that API is exactly what any sane person would want, but AWS didn't do that, which is why I'm asking the question".

Why are you so sure it's what someone sane would want? Maybe there are other ways because there are hidden problems and edge cases with that procedure. It could contradict the fundamental model of the underlying resources but looks correct to someone with a cursory understanding.

I'm not saying this is the case, but LLMs are often wrong in subtle ways like this.

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#53

Post by copperroof »

troupo wrote: post #41
pinoy420 wrote: post #32

Earlier quoted context omitted.

A good prompt. You don’t just ask it. You tell it how to behave and give it a shot load of context

Doesn't prevent it from hallucinating, only reduces hallucinations by a single digit percentage

Personally I’ve been finding that the more context I provide the more it hallucinates.

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#55

Post by dijksterhuis »

Lionga wrote: post #21
Chance-Device wrote: post #3

It’s not really hallucinating though, is it? It’s repeating a pattern in its training data, which is wrong but is presented in that training data (and by the author of this piece, but unintentionally) as being the solution to the problem. So this has more in common with an attack than a hallucination on the LLM’s part.

So nothing is a hallucination ever, because anything a LLM ever spits out is somehow somewhere in the training data?

Technically it's the other way around. All LLMs do is hallucinate based on the training data + prompt. They're "dream machines". Sometimes those "dreams" might be useful (close to what the user asked for/wanted). Oftentimes they're not.

> to quote karpathy: "I always struggle a bit with I'm asked about the "hallucination problem" in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines."

https://nicholas.carlini.com/writing/2025/forecasting-ai-202... (click the button to see the study then scroll down to the hallucinations heading)

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#56

Post by Chance-Device »

do_not_redeem wrote: post #12
Chance-Device wrote: post #3

It’s not really hallucinating though, is it? It’s repeating a pattern in its training data, which is wrong but is presented in that training data (and by the author of this piece, but unintentionally) as being the solution to the problem. So this has more in common with an attack than a hallucination on the LLM’s part.

So anyone can make up some random syntax/fact and post it once, and in some cases the model will take it as truth. I don't know if there's a widely agreed-on definition of "hallucination", but if this isn't one, then the distinction is meaningless imo.

I’m going to double down on this one: an LLM is only as good as its training data. A hallucination to me is an invented piece of information, here it’s going on something real that it’s seen. To me that’s at best contamination, at worst an adversarial attack - something that’s been planted in the data. Here this is obviously not the case, which is why I said “more in common with” instead of “is” above.

Semantics perhaps, but that’s my take.

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#57

Post by MaxikCZ »

Terr_ wrote: post #47
kgeist wrote: post #35

Earlier quoted context omitted.

I use LLMs for writing generic, repetitive code, like scaffolding. It's OK with boring, generic stuff. Sure it makes mistakes occasionally but usually it's a no-brainer to fix them.

> I use LLMs for writing generic, repetitive code, like scaffolding. It's OK with boring, generic stuff. In other words, they're OK in use-cases that programmers need to eliminate , because it means there's high demand for a reusable library, some new syntax sugar, or an improved API.

You can use niche libraries and still benefit from AI, basically anything I want to code is something that I don't think exists, and to bend API in a way that makes a generic library do just what I want means basically writing the same code, just instead of contributing to library its immediate, because the "piping" around it magically appears. I bet if it keeps improving at a current late for a decade, the occupation "programmer" will morph into "architect".

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#60

Post by bakugo »

andix wrote: post #30

I've got a lot of hallucinations like that from LLMs. I really don't get how so many people can get LLMs to code most of their tasks without those issues permanently popping up.

> I really don't get how so many people can get LLMs to code most of their tasks without those issues permanently popping up

They can't, they usually just don't understand the code enough to notice the issues immediately.

The perceived quality of LLM answers is inversely proportional to the user's understanding of the topic they're asking about.

Post reply on HN