Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Making o1, o3, and Sonnet 3.7 hallucinate for everyone

bengarcia.dev

Post reply on HN
Print view View original

31–40 of 233 posts

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#31

Post by skissane »

svantana wrote: post #27
jurgenaut23 wrote: post #18

Earlier quoted context omitted.

Well, it makes sense. The smaller the niche, the lesser weight in the overall training loss. At the end of the day, LLMs are (literally) classifiers that assign probabilities to tokens given some previous tokens.

Yes, but o1, o3 and sonnet are not necessarily pure language models - they are opaque services. For all we know they could do syntax-aware processing or run compilers on code behind the scenes.

The fact they make mistakes like this implies they probably don’t, since surely steps like that would catch many of these

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#32

Post by pinoy420 »

andix wrote: post #30

I've got a lot of hallucinations like that from LLMs. I really don't get how so many people can get LLMs to code most of their tasks without those issues permanently popping up.

A good prompt. You don’t just ask it. You tell it how to behave and give it a shot load of context

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#33

Post by _cs2017_ »

Chance-Device wrote: post #3

It’s not really hallucinating though, is it? It’s repeating a pattern in its training data, which is wrong but is presented in that training data (and by the author of this piece, but unintentionally) as being the solution to the problem. So this has more in common with an attack than a hallucination on the LLM’s part.

Nope there's no attack here.

The training data is the Internet. It has mistakes. There's no available technology to remove all such mistakes.

Whether LLMs hallucinate only because of mistakes in the training data or whether they would hallucinate even if we removed all mistakes is an extremely interesting and important question.

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#34

Post by doubletwoyou »

nopurpose wrote: post #17
dominicq wrote: post #5

ChatGPT used to assure me that you can use JS dot notation to access elements in a Python dict. It also invented Redocly CLI flags that don't exist. Claude sometimes invents OpenAPI specification rules. Any time I ask anything remotely niche, LLMs are often bad.

It tried to convince me that it is possible to break out of outer loop in C++ with `break 'label` statement placed in nested loop. No such syntax exists.

The funny thing is that I think that’s a feature in D.

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#35

Post by kgeist »

andix wrote: post #30

I've got a lot of hallucinations like that from LLMs. I really don't get how so many people can get LLMs to code most of their tasks without those issues permanently popping up.

I use LLMs for writing generic, repetitive code, like scaffolding. It's OK with boring, generic stuff. Sure it makes mistakes occasionally but usually it's a no-brainer to fix them.

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#36

Post by andix »

kgeist wrote: post #35
andix wrote: post #30

I've got a lot of hallucinations like that from LLMs. I really don't get how so many people can get LLMs to code most of their tasks without those issues permanently popping up.

I use LLMs for writing generic, repetitive code, like scaffolding. It's OK with boring, generic stuff. Sure it makes mistakes occasionally but usually it's a no-brainer to fix them.

I try to keep "boring" code to a minimum, by finding meaningful and simple abstractions. LLMs are especially bad handling those, because they were not trained on non-standard abstractions.

Edit: most LLMs are great for spitting out some code that fulfills 90% of what you asked for. That's sometimes all you need. But we all know that the last 10% usually take the same amount of effort as the first 90%.

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#38

Post by x______________ »

ljm wrote: post #19
dominicq wrote: post #5

ChatGPT used to assure me that you can use JS dot notation to access elements in a Python dict. It also invented Redocly CLI flags that don't exist. Claude sometimes invents OpenAPI specification rules. Any time I ask anything remotely niche, LLMs are often bad.

I once asked Perplexity (using Claude underneath) about some library functionality, which it totally fabricated. First, I asked it to show me a link to where it got that suggestion, and it scolded me saying that asking for a source is problematic and I must be trying to discredit it. Then after I responded to that it just said “this is what I thought a solution would look like because I couldn’t find what you were as…

I concur and can easily see this occurring in several areas, for example with Linux troubleshooting. I recently found myself going down a rabbit hole of ever-increasing complicated troubleshooting steps with command that didn't exist, and after several hours of trial and error, gave up after considering the next steps brick-worthy of the system..

Dgg'ing google is still a better resort despite the drop in quality results.

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#39

Post by Yoric »

nopurpose wrote: post #17
dominicq wrote: post #5

ChatGPT used to assure me that you can use JS dot notation to access elements in a Python dict. It also invented Redocly CLI flags that don't exist. Claude sometimes invents OpenAPI specification rules. Any time I ask anything remotely niche, LLMs are often bad.

It tried to convince me that it is possible to break out of outer loop in C++ with `break 'label` statement placed in nested loop. No such syntax exists.

Sounds like it's confusing C++ and Rust. To be fair, their syntaxes are rather similar.

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#40

Post by andix »

pinoy420 wrote: post #32
andix wrote: post #30

I've got a lot of hallucinations like that from LLMs. I really don't get how so many people can get LLMs to code most of their tasks without those issues permanently popping up.

A good prompt. You don’t just ask it. You tell it how to behave and give it a shot load of context

With Claude the context window is quite small. But with adding too much context it often seems to get worse. If the context is not carefully narrowly picked and too unrelated, the LLMs often start to do unrelated things to what you've asked.

At some point it's not really worth anymore creating the perfect prompt, just code it yourself. Also saves the time to carefully review the AI generated code.

Post reply on HN