Live data from Hacker News

Adding a feature because ChatGPT incorrectly thinks it exists

holovaty.com

271–280 of 451 posts

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#271
post #169

Earlier quoted context omitted.

> Sometimes it comes up with a better approach than I had thought of. IMO this has always been the killer use case for AI—from Google Maps to Grammarly. I discovered Grammarly at the very last phase of writing my book. I accepted maybe 1/3 of its suggestions, which is pretty damn good considering my book had already been edited by me dozens of times AND professionally copy-edited. But if I'd have accepted all of Gram…

> The problem is executives want to completely remove humans from the loop, which almost universally leads to disastrous results Thanks for your words of wisdom, which touch on a very important other point I want to raise: often, we (i.e., developers, researchers) construct a technology that would be helpful and "net benign" if deployed as a tool for humans to use, instead of deploying it in order to replace humans.…

The problem here is societal, not technological. An end state where people do less work than they do today but society is more productive is desirable, and we shouldn't be trying to force companies/governments/etc to employ people to do an unnecessary job.

The problem is that people who are laid off often experience significant life disruption. And people who work in a field that is largely or entirely replaced by technology often experience permanent disruption.

However, there's no reason it has to be this way - the fact people having their jobs replace by technology are completely screwed over is a result of the society we have all created together, it's not a rule of nature.

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#272
post #254

I think folks have taken the wrong lesson from this. It’s not that they added a new feature because there was demand. They added a new feature because technology hallucinated a feature that didn’t exist. The savior of tech, generative AI, was telling folks a feature existed that didn’t exist. That’s what the headline is, and in a sane world the folks that run ChatGPT would be falling over themselves to be sure it did…

Sometimes you just have to deal with the world as it is, not how you think it should be.

Is it your argument that the folks that make generative AI applications have nothing to improve from this example?

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#273
post #257
post #207

Earlier quoted context omitted.

That doesn't make sense. It's like saying that the best general advice about which way to turn when you're driving is to turn right. From your comment at https://news.ycombinator.com/item?id=44493308 , and from the fact that you used the passive voice in your comment ("they should be avoided") apparently without noticing, it appears that the reason you have this opinion is that you don't know what the passive voice i…

> the best general advice about which way to turn At the risk of derailing into insane pedantry land, this part is kinda true, so maybe not the best analogy? From routing efficiency: https://www.ge.com/news/reports/ups-drivers-dont-turn-left-p... And also safety: https://www.phly.com/rms/blog/turning-left-at-an-intersectio...

I cherish your pedantry. If not here, where?

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#274
post #254

I think folks have taken the wrong lesson from this. It’s not that they added a new feature because there was demand. They added a new feature because technology hallucinated a feature that didn’t exist. The savior of tech, generative AI, was telling folks a feature existed that didn’t exist. That’s what the headline is, and in a sane world the folks that run ChatGPT would be falling over themselves to be sure it did…

> in a sane world the folks that run ChatGPT would be falling over themselves to be sure it didn’t happen again

This would be a world without generative AI available to the public, at the moment. Requiring perfection would either mean guardrails that would make it useless for most cases, or no LLM access until AGI exists, which are both completely irrational, since many people are finding practical value in its current imperfect state.

The current state of LLM is useful for what it's useful for, warnings of hallucinations are present on every official public interface, and its limitations are quickly understood with any real use.

Nearly everyone in AI research is working on this problem, directly or indirectly.

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#275
post #254

I think folks have taken the wrong lesson from this. It’s not that they added a new feature because there was demand. They added a new feature because technology hallucinated a feature that didn’t exist. The savior of tech, generative AI, was telling folks a feature existed that didn’t exist. That’s what the headline is, and in a sane world the folks that run ChatGPT would be falling over themselves to be sure it did…

[dead]

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#276
post #274
post #254

I think folks have taken the wrong lesson from this. It’s not that they added a new feature because there was demand. They added a new feature because technology hallucinated a feature that didn’t exist. The savior of tech, generative AI, was telling folks a feature existed that didn’t exist. That’s what the headline is, and in a sane world the folks that run ChatGPT would be falling over themselves to be sure it did…

> in a sane world the folks that run ChatGPT would be falling over themselves to be sure it didn’t happen again This would be a world without generative AI available to the public, at the moment. Requiring perfection would either mean guardrails that would make it useless for most cases, or no LLM access until AGI exists, which are both completely irrational, since many people are finding practical value in its curre…

No one is “requiring perfection”, but hallucination is a major issue and is in the opposite direction of the “goal” of AGI.

If “don’t hallucinate” is too much to ask then ethics flew out the window long ago.

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#277
post #276
post #274

Earlier quoted context omitted.

> in a sane world the folks that run ChatGPT would be falling over themselves to be sure it didn’t happen again This would be a world without generative AI available to the public, at the moment. Requiring perfection would either mean guardrails that would make it useless for most cases, or no LLM access until AGI exists, which are both completely irrational, since many people are finding practical value in its curre…

No one is “requiring perfection”, but hallucination is a major issue and is in the opposite direction of the “goal” of AGI. If “don’t hallucinate” is too much to ask then ethics flew out the window long ago.

> No one is “requiring perfection”

> If “don’t hallucinate” is too much to ask then ethics flew out the window long ago.

Those sentences aren't compatible.

> but hallucination is a major issue

Again, every official public AI interface has warnings/disclaimers for this issue. It's well known. It's not some secret. Every AI researcher is directly or indirectly working on this.

> is in the opposite direction of the “goal” of AGI

This isn't a logical statement, so it's difficult to respond to. Hallucination isn't a direction that's being headed towards, it's being actively, with intent and $$$, headed away from.

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#278

I wrote this the other day: > Hallucinations can sometimes serve the same role as TDD. If an LLM hallucinates a method that doesn’t exist, sometimes that’s because it makes sense to have a method like that and you should implement it. — https://www.threads.com/@jimdabell/post/DLek0rbSmEM I guess it’s true for product features as well.

> I wrote this the other day:

>> Hallucinations can sometimes serve the same role as TDD. If an LLM hallucinates a method that doesn’t exist, sometimes that’s because it makes sense to have a method like that and you should implement it.

A detailed counterargument to this position can be found here[0]. In short, what is colloquially described as "LLM hallucinations" do not serve any plausible role in software design other than to introduce an opportunity for software engineers to stop and think about the problem being solved.

See also Clark's third law[1].

0 - https://addxorrol.blogspot.com/2025/07/a-non-anthropomorphiz...

1 - https://en.wikipedia.org/wiki/Clarke%27s_three_laws

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#279
post #37

I wrote this the other day: > Hallucinations can sometimes serve the same role as TDD. If an LLM hallucinates a method that doesn’t exist, sometimes that’s because it makes sense to have a method like that and you should implement it. — https://www.threads.com/@jimdabell/post/DLek0rbSmEM I guess it’s true for product features as well.

Seems like lots of us have stumbled on this. It’s not the worst way to dev! > Maybe hallucinations of vibe coders are just a suggestion those API calls should have existed in the first place. > Hallucination-driven-development is in. https://x.com/pwnies/status/1922759748014772488?s=46&t=bwJTI...

Beware, the feature in OP isn't something that people would have found useful, it's not like chatgpt assigned to OP's business a request from a user in some latent consumer-provider space, as if chatgpt were some kind of market maker connecting consumers with products, like a google with organic content or ads, or linkedin or producthunt.

No, what actually happened is that OP developed a type of chatgpt integration, and a shitty one at that, chatgpt could have just directed the user to the site and told them to upload that image to OP's site. But it felt it needed to do something with the image, so it did.

There's no new value add here, at least yet, maybe if users started requesting changes to the sheet I guess, not what's going on.

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#280
TDD meets LLM-driven API design.

I recall that early on a coworker was saying that ChatGPT hallucinated a simpler API than the one we offered, albeit with some easy to fix errors and extra assumptions that could've been nicer defaults in the API. I'm not sure if this ever got implemented though, as he was from a different team.

Post reply on HN