Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why does no one seem to care that AI gives wrong answers?

news.ycombinator.com

Post reply on HN
Print view View original

111–119 of 119 posts

Re: Ask HN: Why does no one seem to care that AI gives wrong answers?

#111

Post by yaj54 »

tivert wrote: post #86
yaj54 wrote: post #59

LLMs would be better nomenclature than AI in this context. LLMs are not factual databases. They are not trained to retrieve or produce factual statements. LLMs give you the most likely word after some prior words. They are incredibly accurate at estimating the probabilities of the next word. It is a weird accident that you can use auto-regressive next word prediction to make a chat bot. It's even weirder that you can…

> LLMs are best thought of as language generators (or "writers") not as repositories of knowledge and facts. And the utility of a "language generator" without reliable knowledge or facts is extremely limited. The technical term for that kind of language is bullshit. > People don't care about (or are willing to accept) the "wrong answers" because there are enough use cases for "writing" that don't require factual accu…

I'd argue that what you're talking about in fiction is coherence (internal consistency) not factual accuracy (consistency with an externally verifiably ground truth).

I'd also argue that the economic value of coherent bullshit is ... quite high. Many people have made careers out of producing coherent bullshit (some even with incoherent bullshit :-).

Of course, in the long run, factual accuracy has more economic value than bullshit.

Re: Ask HN: Why does no one seem to care that AI gives wrong answers?

#112

Post by rossdavidh »

stavros wrote: post #108
rossdavidh wrote: post #105

Earlier quoted context omitted.

I'm sure there's plenty I haven't realized, but the reason it's worth pointing out that LLM's are not intelligent, is that their boosters routinely refer to them as "AI", and the "I" in there stands for "intelligence", so pointing out that the label applied is not accurate, is important.

In that case, what AI system do you feel that the label can be applied to?

I don't think there is one. Many researchers had tried to switch the field to using "ML" instead, since it's a lot more accurate of a label, but it doesn't hype as well, and that appears to have been the decisive factor.

Re: Ask HN: Why does no one seem to care that AI gives wrong answers?

#113

Post by stavros »

rossdavidh wrote: post #112
stavros wrote: post #108

Earlier quoted context omitted.

In that case, what AI system do you feel that the label can be applied to?

I don't think there is one. Many researchers had tried to switch the field to using "ML" instead, since it's a lot more accurate of a label, but it doesn't hype as well, and that appears to have been the decisive factor.

When I did my masters fifteen years ago, it was a masters in "machine learning". AI was already just what laypeople called it.

If your argument is that "this isn't AGI", I don't think anyone at all disagrees, but then that's a bit of a tautology.

Re: Ask HN: Why does no one seem to care that AI gives wrong answers?

#114

Post by a3n »

paulmd wrote: post #100
a3n wrote: post #62

Earlier quoted context omitted.

> I was already using it in that context so I pushed back and it said "Oh yeah you're right, sorry" Same thing happened to me. I asked for all the Ukrainian noun cases, it listed and described six. I responded that there are seven. "Oh, right." It then named and described the seventh. That's no better than me taking an exam, so why should I rely on it, or use it at all?

If you find it absolutely necessary to only work with coworkers who are incapable of making mistakes, I assume that you probably work alone?

Actually I do mostly work alone, I'm a truck driver. But that has nothing to do with my scepticism.

Re: Ask HN: Why does no one seem to care that AI gives wrong answers?

#115

Post by tivert »

yaj54 wrote: post #111
tivert wrote: post #86

Earlier quoted context omitted.

> LLMs are best thought of as language generators (or "writers") not as repositories of knowledge and facts. And the utility of a "language generator" without reliable knowledge or facts is extremely limited. The technical term for that kind of language is bullshit. > People don't care about (or are willing to accept) the "wrong answers" because there are enough use cases for "writing" that don't require factual accu…

I'd argue that what you're talking about in fiction is coherence (internal consistency) not factual accuracy (consistency with an externally verifiably ground truth). I'd also argue that the economic value of coherent bullshit is ... quite high. Many people have made careers out of producing coherent bullshit (some even with incoherent bullshit :-). Of course, in the long run, factual accuracy has more economic value…

> I'd argue that what you're talking about in fiction is coherence (internal consistency) not factual accuracy (consistency with an externally verifiably ground truth).

No. I'm talking about "factual accuracy (consistency with an externally verifiably ground truth)." Mere internal consistency is not enough: a fictional world where everyone consistently stabs themselves in the eye when they see flashing lights is consistent, but lacks factual accuracy, and is therefore garbage fiction.

> I'd also argue that the economic value of coherent bullshit is ... quite high. Many people have made careers out of producing coherent bullshit (some even with incoherent bullshit :-).

I agree there's (greedily selfish) "economic value" to coherent bullshit, but there's negative social value to it. It's basically a kind of scam.

IMHO, some of the best applications for LLMs are for things like spam and scams, not the utopian BS they're promoted for (e.g. some LLM will diagnose your illness better, faster, and cheaper than a doctor).

Re: Ask HN: Why does no one seem to care that AI gives wrong answers?

#116

Post by thiht »

Honestly I find LLMs to be a great tool, when using them right, and with the sufficient skills to know when they’re wrong. And for some problems, you don’t need a 100% right answer.

Earlier today I asked ChatGPT to give me a Go script to parse a Go codebase (making heavy use of Go AST libraries which I never use normally) and it gave me a 90% good solution which saved me a lot of time. To be clear the solution was non functional on its own, but it still saved me from doing exploration work and gave me a quick overview of the APIs I would need.

A few days ago it helped me generate code for some obscure AWS API using aws-sdk-go-v2. It was again almost fully working, and better than the examples I could find online.

I have examples like this every week. It’s not as amazing as some people say, but still pretty useful. I rejected AI stuff at first but don’t regret adding LLMs to my toolbelt.

Re: Ask HN: Why does no one seem to care that AI gives wrong answers?

#117

Post by ThunderSizzle »

latentsea wrote: post #84
ThunderSizzle wrote: post #30

Earlier quoted context omitted.

Assuming they aren't convinced they are wrong - and multiple opinions can be valid while differing. The world isnt always black and white. An AI can never be convinced it's wrong permanently, and sometimes it can't be convinced ed temporarily, depending on the model

Although new models do get trained and replace older ones, so from a users perspective it's not like they'll never change their answers on things. We've seen improvements over time, so while individual models are relatively fixed,the LLM industry itself is much more dynamic.

Yes, but the annoying part to me is that the model doesn't enhance itself based on my history with it.

Re: Ask HN: Why does no one seem to care that AI gives wrong answers?

#118

Post by mmusson »

yaj54 wrote: post #59

LLMs would be better nomenclature than AI in this context. LLMs are not factual databases. They are not trained to retrieve or produce factual statements. LLMs give you the most likely word after some prior words. They are incredibly accurate at estimating the probabilities of the next word. It is a weird accident that you can use auto-regressive next word prediction to make a chat bot. It's even weirder that you can…

The thing I find fascinating is that apparently there is a chunk of behavior that we might define as “intelligent” on some level that seems directly encoded in language itself.

Re: Ask HN: Why does no one seem to care that AI gives wrong answers?

#119

Post by yaj54 »

mmusson wrote: post #118
yaj54 wrote: post #59

LLMs would be better nomenclature than AI in this context. LLMs are not factual databases. They are not trained to retrieve or produce factual statements. LLMs give you the most likely word after some prior words. They are incredibly accurate at estimating the probabilities of the next word. It is a weird accident that you can use auto-regressive next word prediction to make a chat bot. It's even weirder that you can…

The thing I find fascinating is that apparently there is a chunk of behavior that we might define as “intelligent” on some level that seems directly encoded in language itself.

I completely agree. As language is the preferred encoding method for intelligent thought (at least in our species) it could very well be that a sufficiently accurate language model is also a generally intelligent model.
Post reply on HN