I work in AI product eng for a larger company. The honest answer is that with good RAG and few-shot prompting, we can consider actual incorrect output to be a serious and reproducible bug. This means that when we call LLMs in production, we get about the same wrong-answer rate as we do any other kind of product engineering bug.
Ask HN: Why does no one seem to care that AI gives wrong answers?
31–40 of 119 posts
Re: Ask HN: Why does no one seem to care that AI gives wrong answers?
#32I’ll be honest, every day people tell me the magical things all these new AI tools can do and I try them and usually find the results useless.
Every AI chatbot I’ve ever interacted with has been unable to help me. The things I’ve had them write do usually pass the Turing Test, but are rarely even close to as good as what I could write myself. (I admit, being self-employed for a long time, I can just avoid a lot of busy work that many people cannot, so I may be missing lots of great use cases there. I never find myself having to write something that isn’t great and wanting to just get it over with. AI might be great if you do. )
I’ve been trying to use image/video creation to do lots of other things and I’ve not even come close to getting anything usable.
I appreciate certain things (ability to summarize, great voice to text transcription, etc.) but find a lot of it to be not very useful and overhyped in its current form.
Re: Ask HN: Why does no one seem to care that AI gives wrong answers?
#33I didn't care too much at first because it seemed that the rate of improvement was sufficient to cover a multitude of sins. But I'm starting to, because it is becoming clear that progress hit an absolute brick wall with GPT4. If anything it has has gone backward since then.
Just today, ChatGPT4o screwed up a rudimentary arithmetic problem ( https://i.imgur.com/2jNXPBF.png ) that I'd swear the previous GPT4 model would have gotten right.
And then there's this shitshow: https://news.ycombinator.com/item?id=40894167 Which is still happening as of this morning, only now all my previous history is gone. Nothing left but links to other peoples' chats. If someone at OpenAI still cares what they are doing, it's not obvious.
Re: Ask HN: Why does no one seem to care that AI gives wrong answers?
#34I’m also curious about this. This morning I needed to generate summaries of news articles when I noticed Bing AI was inserting facts that weren’t present in the source article. (Not correct at all) It really hurts the potential of what AI could do if I have to double check everything it generates. We wouldn’t accept a spreadsheet program that required double checking with a calculator, why do LLMs get a pass?
With Excel you might want that calculator.
Re: Ask HN: Why does no one seem to care that AI gives wrong answers?
#35It can give wrong answers sometimes and still be useful. Also there are many tasks where it almost always gives correct answers, text to speech with function calling is 100x better now than it was 2 years ago. And in some spaces, correctness is a fuzzy concept anyway (creative spaces).
Re: Ask HN: Why does no one seem to care that AI gives wrong answers?
#36Personally, I and people I've spoken with use LLMs less and less because of how often they're wrong. The other day I asked ChatGPT about a specific built-in method in Java and it told me that it couldn't do one specific thing. I was already using it in that context so I pushed back and it said "Oh yeah you're right, sorry"
I feel like I can't trust anything it says. Mostly I use it to parse things I don't understand and then do my own verification that it's correct.
All that to say, from my perspective, they're losing some small amount of ground. The other side is that the big corps that run them don't want their golden gooses to be cooked. So they keep pushing them and shoving them into everything unnecessarily and we just have to eat it.
So I think it's a perception thing. The corps want us to think it's super useful so it continues to give them record profits. While the rest of us are slowly waking up to how useless they are if they will confidently tell us incorrect answers and are moving away from it.
So you may just be seeing sleezy marketing at work here.
Re: Ask HN: Why does no one seem to care that AI gives wrong answers?
#37To paraphrase Upton Sinclair, it is difficult to get a man to care about something, when his salary depends on his not caring about it.
A lot of money has poured into AI, money potentially well in excess of the return on investment over the next several years. The field, from investors to CEOs and downwards to developers, is in a state of collective suspension of disbelief. There is going to be a lot of people out of work when reality reasserts itself.
Re: Ask HN: Why does no one seem to care that AI gives wrong answers?
#38> investors/founders/PMs don’t really care
Garry Tan from YC is a great example of this.
It's not that he doesn't care. It's just that he believes that the next model will be the one that fixes it. And companies that jump on board now can simply update their model and be in prime position. Similar to how Tesla FSD is always 2 weeks away from perfection and when it happens they will dominate the market.
And because companies are experimenting with how to apply AI these startups are making money. So investors jump in on the optimism.
The problem is that for many use cases e.g. AI agents, assistance, search, process automation etc. they very much do care about accuracy. And they are starting to run out of patience for the empty promises. So there is a reckoning coming for AI in the coming year or two and it will be brutal. Especially in this fundraising environment.
Re: Ask HN: Why does no one seem to care that AI gives wrong answers?
#39Hype. Every so many years some sort of semi-novel software gets invented/improved and some dude puts 100mg of 2c-b in his Huel shake and realizes that he’s invented the godhead. This dude invariably has buddies that do VC.
It’s the same reason why we heard about blockchain for years despite it having near zero practical uses
Re: Ask HN: Why does no one seem to care that AI gives wrong answers?
#40Lots of people care. From a coding perspective, proper technical systems already have checks and balances (e.g. test cases) to catch bad code, and is something that's important to have regardless of generative AI usage. From a creative/informational perspective, there are stories every day of hallucinations and the tech companies are correctly dunked on because of it. That's more product management error than AI erro…
The fact that every AI-based company gets dunked-on for hallucination somewhat suggests that hallucination is a showstopper issue and in fact cannot be worked around.
I agree personally. I don't use LLMs for most things these days because I've been bitten in the ass enough times (whether it was real fallout or just me being able to prove the LLM wrong) that I don't trust them at all. To me it's a showstopper because it can get even the simplest explanations very incorrect and that's not useful to me.
They're still my first go over google these days but I usually only use them for code or weird word transformations and any information is double checked. Pretty useless as an answer bot.