Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Making o1, o3, and Sonnet 3.7 hallucinate for everyone

bengarcia.dev

Post reply on HN
Print view View original

221–230 of 233 posts

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#221

Post by Kye »

martin-t wrote: post #46
asadotzler wrote: post #10

Earlier quoted context omitted.

Everything they do is hallucination, some of it ends up being useful and some of it not. The not useful stuff gets called confabulation or hallucination but it's no different from the useful stuff, generated the same exact way. It's all bullshit. Bullshit is actually useful though, when it's not so wrong that it steers people wrong.

More people need to understand this. There was an article that explained it concisely but i can't find anymore (and of course LLMs are not helpful in this because they don't work well when you want them to retrieve actual information)

It probably wasn't mine[0], but this is how I tend to put it:

>> "The more you can see the inputs and outputs as blobs of "stuff," the better. If LLMs think, it's not in any way we yet understand. They're probability engines that transform data into different data using weighted probabilities."

Stuff in, stuff out.

[0] https://kyefox.com/ai-assisted-creativity/

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#223

Post by visarga »

latexr wrote: post #219
visarga wrote: post #203

Earlier quoted context omitted.

> All they do is regurgitate text which resembles a continuation of what came before, and sometimes—but with zero guarantees—that text aligns with reality. You mean like us? Because it takes many runs and debug rounds to make anything that works. Can you write a complex code top-to-bottom in one round, or do you gradually test it to catch bugs you have "hallucinated"? Both humans and LLMs get into bugs, the question…

> You mean like us? Not, not like us. This constant comparison of LLMs to humans is tiresome and unproductive. I’m not a proponent of human exceptionalism, but pretending LLMs are on par is embarrassing. If you want to claim your thinking ability isn’t any better than an LLM, that’s your prerogative, but the dullest human I’m acquainted with is still capable of remembering and thinking through in a way no LLM does. H…

They are on par on many tasks, surpass us in some tasks, and catching up on the rest quite fast. Both humans and LLMs forget details, both have to go through iterative bug fixing when creating something. Longer contexts and memory are coming, they already exist but need improvement.

> I have never “hallucinated” a feature

You never mistook an argument, or forgot one of the 100 details we have to mind while writing complex apps?

I would rather measure humans vs AI not in basic skill capability but in authonomy. I think AI still has much more to catch up on that front.

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#224

Post by derefr »

anonzzzies wrote: post #206
AprilisKalends wrote: post #174

Earlier quoted context omitted.

I personally haven't figured out why there isn't a tool that just loops the AI severals times on a task by compiling, feeding in the errors adjusting and repeating and then letting the user review the result be in a success or failure by exceed the loop limits.

The LLMs want to please, they can infinitely keep making 'important' changes; changing variable names without changing anything else, adding/removing comments, adding removing print statements, moving a function somewhere in the code (usually by first duplicating it) and fixing perfectly fine code with 'let me see if everything is really working' (it was and now it's broken again).

"Wanting to please" is just fine-tuned implicit prompt stuff, though. You can tell the LLM that it's roleplaying as a senior engineer, and should consider you, the human to be a junior engineer it works with, who is asking it for advice. You can just leave it at that (often works, if the AI knows enough about how software is produced); or you can describe particular traits of senior engineers — e.g. "wants code that works" and "knows when to say the code is good enough and that the junior is now wasting time by fiddling with it."

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#225

Post by wrs »

miningape wrote: post #52
wrs wrote: post #42

Earlier quoted context omitted.

This sort of hallucination happens to me frequently with AWS infrastructure questions. Which is depressing because I can't do anything but agree, "yeah, that API is exactly what any sane person would want, but AWS didn't do that, which is why I'm asking the question".

Why are you so sure it's what someone sane would want? Maybe there are other ways because there are hidden problems and edge cases with that procedure. It could contradict the fundamental model of the underlying resources but looks correct to someone with a cursory understanding. I'm not saying this is the case, but LLMs are often wrong in subtle ways like this.

Well, I do consider myself sane, and it’s what I want. :) But usually it’s a missing higher-level abstraction, where AWS could have given you the Millenium Falcon kit, but instead it just dumps a bunch of LEGO pieces on your desk and tells you to figure it out.

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#226

Post by [deleted] »

AprilisKalends wrote: post #174
derefr wrote: post #166

Earlier quoted context omitted.

I would love an AI coding assistant that doesn't respond until it's gone out and tested its answer in a sandbox to confirm it actually compiles.

I personally haven't figured out why there isn't a tool that just loops the AI severals times on a task by compiling, feeding in the errors adjusting and repeating and then letting the user review the result be in a success or failure by exceed the loop limits.

[deleted]

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#227

Post by rawgreaze »

zeroq wrote: post #152

This is exactly what I mean when I say tell me your bad without saying so. Most people here disagree with that. A while back a friend of mine told me he's very found of llms because he's confused with kubernetes cli and instead of looking up the answer on the internet he can simply state his desire in a chat to get the right answer. Well... Sure, but if you'd look the answer on stackoverflow you'd see the whole threa…

It's more about how you use them. Asking a general-purpose model like ChatGPT precise k8s questions might prove counterproductive; however feeding the entire k8s documentation into an LLM like Gemini and asking questions that way is invaluable. Not just the documentation, but your entire cluster config. Like you said, "blindly trusting LLMs, you'll find yourself in trouble" this is true, but the same can be said for StackOverflow or any other resource. Sifting through StackOverflow to find the exact answer to your question (and then understanding the answer, and hoping that it pertains to your environment, and version etc) is much less efficient when you can ingest the entire docs, your config, your environment, and your question, and have it spit out exactly what you need in whatever format you need. You can even web search with multiple questions derived automatically from your main question, to gather multiple sources which are aggregated and referenced in the answer so you can easily cross-check for hallucinations. StackOverflow isn't even as easy to fact check as LLMs considering the in-line sources.

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#228

Post by int_19h »

Yoric wrote: post #39
nopurpose wrote: post #17

Earlier quoted context omitted.

It tried to convince me that it is possible to break out of outer loop in C++ with `break 'label` statement placed in nested loop. No such syntax exists.

Sounds like it's confusing C++ and Rust. To be fair, their syntaxes are rather similar.

It's confusing C++ in Java - the latter has labelled breaks but no goto.

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#229

Post by johnisgood »

johnisgood wrote: post #118
andix wrote: post #114

Earlier quoted context omitted.

I just looked it up, it seems to be the rate limit that's actually kicking in for me.

Yes, that's it! It is frustrating to me, too. You have to start a new chat with all relevant data, and a detailed summary of the progress/status.

(Because you reach the limit faster otherwise).

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#230

Post by deterministic »

kgeist wrote: post #35
andix wrote: post #30

I've got a lot of hallucinations like that from LLMs. I really don't get how so many people can get LLMs to code most of their tasks without those issues permanently popping up.

I use LLMs for writing generic, repetitive code, like scaffolding. It's OK with boring, generic stuff. Sure it makes mistakes occasionally but usually it's a no-brainer to fix them.

I use a custom code generator for that. It works much better than trying to explain to an AI exactly what I want. Yes it’s a it more work up front but worth it IMHO.
Post reply on HN