Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Making o1, o3, and Sonnet 3.7 hallucinate for everyone

bengarcia.dev

Post reply on HN
Print view View original

181–190 of 233 posts

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#181

Post by [deleted] »

CSSer wrote: post #180
grey-area wrote: post #173

Earlier quoted context omitted.

Where the GAI hallucinates an api it is not always easy to find out if it exists given different versions and libraries for a given task, it can easily waste 10 mins trying to find the promised api, particularly when search results also include generated ai answers. Also there are plenty of mistakes that will compile and give subtle errors, particularly in dynamic languages and those which allow implicit coercion. Ja…

My favorite is when it hallucinates a library that does exist and ostensibly has a relevant title and methods but upon investigation proves to be for a purpose wholly unrelated to the current task. To make matters worse, this discovery will be greeted, every time, by "Ah, you're right!"

[deleted]

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#182

Post by simonw »

CSSer wrote: post #180
grey-area wrote: post #173

Earlier quoted context omitted.

Where the GAI hallucinates an api it is not always easy to find out if it exists given different versions and libraries for a given task, it can easily waste 10 mins trying to find the promised api, particularly when search results also include generated ai answers. Also there are plenty of mistakes that will compile and give subtle errors, particularly in dynamic languages and those which allow implicit coercion. Ja…

My favorite is when it hallucinates a library that does exist and ostensibly has a relevant title and methods but upon investigation proves to be for a purpose wholly unrelated to the current task. To make matters worse, this discovery will be greeted, every time, by "Ah, you're right!"

Yeah, those are pretty frustrating. I've learned not to ask the question "Does library X have feature Y?" because if that feature sounds like a good idea it'll often confidently tell me that it exists when it doesn't.

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#183

Post by johnisgood »

shermantanktop wrote: post #168
johnisgood wrote: post #134

Earlier quoted context omitted.

I know what I want to do and how to do it (expert), so the results are good, for me at least. Of course I have to polish it off here and there.

You answered that more graciously than many would.

This is the reason I pay for Claude. If it weren't making me as productive as I am with it, I would not pay for it.

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#184

Post by johnisgood »

crooked-v wrote: post #149
johnisgood wrote: post #110

Earlier quoted context omitted.

In my case the maximum was ~3k LOC.

That's not a "large" project, it's either a toy or a single-purpose tool.

A single-purpose tool, and libraries, yes.

What is the LOC of cgit? Because I made a lot of changes to it, too, albeit privately. I uploaded most files as project files.

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#185

Post by ta988 »

AprilisKalends wrote: post #174
derefr wrote: post #166

Earlier quoted context omitted.

I would love an AI coding assistant that doesn't respond until it's gone out and tested its answer in a sandbox to confirm it actually compiles.

I personally haven't figured out why there isn't a tool that just loops the AI severals times on a task by compiling, feeding in the errors adjusting and repeating and then letting the user review the result be in a success or failure by exceed the loop limits.

They exist in various forms...

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#186

Post by kubb »

latexr wrote: post #43

> Conclusion > LLMs are really smart most of the time. No, the conclusion is they’re never “smart”. All they do is regurgitate text which resembles a continuation of what came before, and sometimes—but with zero guarantees—that text aligns with reality.

People use the general impressiveness of language produced by humans as a proxy measuring their intelligence, that’s how LLMs became AI.

The amount of code out there lets LLMs learn on examples of a lot of tasks and pass SWE benchmarks. Smart autocomplete and solved problem lookup has value. Even if it doesn’t always work correctly, and doesn’t know what it knows.

And for non-programmers they’re indistinguishable from programmers. They produce working code.

It’s easy to see how a product manager or a designer or even a manager that didn’t code for years in a large company can think they’re almost as good as devs.

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#187

Post by afro88 »

andix wrote: post #30

I've got a lot of hallucinations like that from LLMs. I really don't get how so many people can get LLMs to code most of their tasks without those issues permanently popping up.

Here's the secret to coding with an LLM: don't expect it to get things 100% correct. You will need to fix something almost every time you use it to generate code. Maybe a name here, maybe a calculation, maybe a function signature. And maybe you won't spot the issue until later.

You still "used" an LLM to write the code. And it still saved you time (though depending on the circumstances this can be debatable).

That's why all these people say they use LLMs to write lots of code. They aren't saying it did it 100% without any checking and fixing. They're just saying they used it.

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#188

Post by bubblyworld »

jurgenaut23 wrote: post #18
dominicq wrote: post #5

ChatGPT used to assure me that you can use JS dot notation to access elements in a Python dict. It also invented Redocly CLI flags that don't exist. Claude sometimes invents OpenAPI specification rules. Any time I ask anything remotely niche, LLMs are often bad.

Well, it makes sense. The smaller the niche, the lesser weight in the overall training loss. At the end of the day, LLMs are (literally) classifiers that assign probabilities to tokens given some previous tokens.

SOTA language models are trained with reinforcement learning now too, so the model distributions can be far removed from the underlying data distribution. Not that I think your overall point is wrong (it's obviously a very strong bias), but things are getting more complex with time.

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#189

Post by isaacremuant »

DrammBA wrote: post #139
foundry27 wrote: post #15

Earlier quoted context omitted.

It’s always a touch ironic when AI-generated replies such as this one are submitted under posts about AI. Maybe that’s secretly the the self-reflection feedback loop we need for AGI :)

So strange too, their other comments seem normal, but suddenly they decided to post a gpt comment.

At least one other is LLM generated too, from what I saw.

Re: Making o1, o3, and Sonnet 3.7 hallucinate for everyone

#190

Post by bubblyworld »

andix wrote: post #80
hombre_fatal wrote: post #76

Earlier quoted context omitted.

Boilerplate and plumbing code isn't inherently bad, nor do you improve the codebase by factoring it down to zero with libraries and abstractions. As I've matured as a developer, I've appreciated certain types of boilerplate more and more because it's code that shows up in your git diffs. You don't need to chase down the code in some version of some library to see how something works. Of course, not all boilerplate is…

Boilerplate is better than bad abstractions. But good abstractions are far superior.

Abstractions are like mutations - most of them kill the host =P
Post reply on HN