Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why does no one seem to care that AI gives wrong answers?

news.ycombinator.com

Post reply on HN
Print view View original

101–110 of 119 posts

Re: Ask HN: Why does no one seem to care that AI gives wrong answers?

#101

Post by Jensson »

paulmd wrote: post #100
a3n wrote: post #62

Earlier quoted context omitted.

> I was already using it in that context so I pushed back and it said "Oh yeah you're right, sorry" Same thing happened to me. I asked for all the Ukrainian noun cases, it listed and described six. I responded that there are seven. "Oh, right." It then named and described the seventh. That's no better than me taking an exam, so why should I rely on it, or use it at all?

If you find it absolutely necessary to only work with coworkers who are incapable of making mistakes, I assume that you probably work alone?

Coworkers learn, chatgpt doesn't.

Re: Ask HN: Why does no one seem to care that AI gives wrong answers?

#102

Post by Jensson »

stavros wrote: post #97
rossdavidh wrote: post #6

So, if you dealt with a person who knew all the vocabulary related to a field, and could make well-constructed sentences about that field, and sounded confident, it would almost always mean they had spent a lot of time studying that field. That tends to mean that, although they may occasionally make a mistake, they will usually be correct. People apply the same intuition to LLMs, and because it's not a person (and it…

I find that "intelligence" was a shaky concept to begin with, but LLMs have completely thrown the idea in the trash. When someone says "LLMs are not intelligent", I treat that as a bit of a signal that I shouldn't pay much attention to their other points, because if you haven't realized that you don't have a good definition for intelligence, what else haven't you realized?

> When someone says "LLMs are not intelligent", I treat that as a bit of a signal that I shouldn't pay much attention to their other points, because if you haven't realized that you don't have a good definition for intelligence, what else haven't you realized?

So you have a good definition for "intelligent", and it applies to LLM? Please tell us! And explain how that definition is so infallible that you know that everyone who says LLM aren't intelligent are wrong?

Re: Ask HN: Why does no one seem to care that AI gives wrong answers?

#104

Post by rossdavidh »

rsynnott wrote: post #92
rossdavidh wrote: post #6

So, if you dealt with a person who knew all the vocabulary related to a field, and could make well-constructed sentences about that field, and sounded confident, it would almost always mean they had spent a lot of time studying that field. That tends to mean that, although they may occasionally make a mistake, they will usually be correct. People apply the same intuition to LLMs, and because it's not a person (and it…

Even with a human speaker, that's not a totally safe assumption, and a certain type of fraudster relies heavily on people making this type of assumption (the likes of L Ron Hubbard in particular liked to use the language of expertise for various fields in a nonsensical way, and this is extremely convincing to a certain sort of person). But LLMs might almost have been designed to exploit this particular cognitive bias…

Agreed. The optimistic scenario is that being exposed to so many "hallucinating" LLM's, people will become better at spotting the same thing in humans. But, I admit freely that is just the optimistic scenario.

Re: Ask HN: Why does no one seem to care that AI gives wrong answers?

#105

Post by rossdavidh »

stavros wrote: post #97
rossdavidh wrote: post #6

So, if you dealt with a person who knew all the vocabulary related to a field, and could make well-constructed sentences about that field, and sounded confident, it would almost always mean they had spent a lot of time studying that field. That tends to mean that, although they may occasionally make a mistake, they will usually be correct. People apply the same intuition to LLMs, and because it's not a person (and it…

I find that "intelligence" was a shaky concept to begin with, but LLMs have completely thrown the idea in the trash. When someone says "LLMs are not intelligent", I treat that as a bit of a signal that I shouldn't pay much attention to their other points, because if you haven't realized that you don't have a good definition for intelligence, what else haven't you realized?

I'm sure there's plenty I haven't realized, but the reason it's worth pointing out that LLM's are not intelligent, is that their boosters routinely refer to them as "AI", and the "I" in there stands for "intelligence", so pointing out that the label applied is not accurate, is important.

Re: Ask HN: Why does no one seem to care that AI gives wrong answers?

#106

Post by stavros »

nunez wrote: post #88
danrob wrote: post #58

Earlier quoted context omitted.

This is a common argument to support the usage of LLMs: "Well, humans do it too." We have many rules, regulations, strategies, patterns, and legions of managers and management philosophy for dealing with humans. With humans, they're incorrect sometimes, yes, and we actively work around their failures. We expect humans to develop over time. We expect them to join a profession and give bad answers a lot. As time goes o…

Also, humans can say "I don't know," a skill that seems impossible for LLMs

What's more useful to me is "I'm not sure" or "I'm very sure", but LLMs can't provide you with a level of certainty. They're very sure about everything, including things they make up.

Re: Ask HN: Why does no one seem to care that AI gives wrong answers?

#107

Post by stavros »

Jensson wrote: post #102
stavros wrote: post #97

Earlier quoted context omitted.

I find that "intelligence" was a shaky concept to begin with, but LLMs have completely thrown the idea in the trash. When someone says "LLMs are not intelligent", I treat that as a bit of a signal that I shouldn't pay much attention to their other points, because if you haven't realized that you don't have a good definition for intelligence, what else haven't you realized?

> When someone says "LLMs are not intelligent", I treat that as a bit of a signal that I shouldn't pay much attention to their other points, because if you haven't realized that you don't have a good definition for intelligence, what else haven't you realized? So you have a good definition for "intelligent", and it applies to LLM? Please tell us! And explain how that definition is so infallible that you know that eve…

> So you have a good definition for "intelligent", and it applies to LLM?

No. I feel like that was my whole point.

Re: Ask HN: Why does no one seem to care that AI gives wrong answers?

#108

Post by stavros »

rossdavidh wrote: post #105
stavros wrote: post #97

Earlier quoted context omitted.

I find that "intelligence" was a shaky concept to begin with, but LLMs have completely thrown the idea in the trash. When someone says "LLMs are not intelligent", I treat that as a bit of a signal that I shouldn't pay much attention to their other points, because if you haven't realized that you don't have a good definition for intelligence, what else haven't you realized?

I'm sure there's plenty I haven't realized, but the reason it's worth pointing out that LLM's are not intelligent, is that their boosters routinely refer to them as "AI", and the "I" in there stands for "intelligence", so pointing out that the label applied is not accurate, is important.

In that case, what AI system do you feel that the label can be applied to?

Re: Ask HN: Why does no one seem to care that AI gives wrong answers?

#109

Post by AnimalMuppet »

paulmd wrote: post #100
a3n wrote: post #62

Earlier quoted context omitted.

> I was already using it in that context so I pushed back and it said "Oh yeah you're right, sorry" Same thing happened to me. I asked for all the Ukrainian noun cases, it listed and described six. I responded that there are seven. "Oh, right." It then named and described the seventh. That's no better than me taking an exam, so why should I rely on it, or use it at all?

If you find it absolutely necessary to only work with coworkers who are incapable of making mistakes, I assume that you probably work alone?

If you have a coworker who makes mistakes at the same rate that ChatGPT makes them, it might be perfectly reasonable for you not to want to work with that coworker.

Humans aren't perfect. But "makes some mistakes" and "confidently spews errors at a high rate" are not the same. The difference matters.

Re: Ask HN: Why does no one seem to care that AI gives wrong answers?

#110

Post by kardos »

The LLMs are basically useless for anything requiring reasoning or to help solve real problems, but the big use-case is that LLMs are competing with google search. Google search is so polluted with low quality ads-first SEO garbage that the may-be-hallucinating-LLM is a more effective way to find some sorts of information than manually slogging through the garbage and possibly getting the wrong answer anyway.

I suppose that there is also some hope that the hallucination problem will erode as more effort/compute is poured into the training. There may need to be a paradigm shift though, the current structure around generating tokens based on probabilities seems like it will forever be a 'regurgitator'.

Post reply on HN