Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are steroids for your Dunning-Kruger

bytesauna.com

Post reply on HN
Print view View original

61–70 of 308 posts

Re: LLMs are steroids for your Dunning-Kruger

#61

Post by simianwords »

>How often do you think a ChatGPT user walks away not just misinformed, but misinformed with conviction? I would bet this happens all the time. And I can’t help but wonder what the effects are in the big picture.

this is so wrong! i simply can't get ChatGPT to admit something clearly wrong. it can play both sides and gives nuance which is exactly what i expect. but it is so un-sycopanthic that it won't leave you feeling like you are right. any examples of it doing so are welcome! show me examples where it takes a clearly wrong or false idea and makes it look as if it is a good idea (unless you specifically ask it to do it).

Re: LLMs are steroids for your Dunning-Kruger

#62

Post by defgeneric »

balderdash wrote: post #16

I ascribe the effect of LLMs as similar to reading the newspaper, when I learn about something I have no knowledge base in I come away feeling like I learned a lot. When I interact with a newspaper or LLM in an area where I have real domain expertise I realize they don’t know what they are talking about - which is concerning about the information I get from them about topics I don’t have that high level of domain exp…

Also known as the "Gell-Mann amnesia effect" [1].

[1] https://en.wikipedia.org/wiki/Gell-Mann_amnesia_effect

Re: LLMs are steroids for your Dunning-Kruger

#64

Post by zkmon »

>> How often do you think a ChatGPT user walks away not just misinformed, but misinformed with conviction? I would bet this happens all the time.

Why is the "Dunning-Kruger" is not mentioned anywhere in the article body while is gloriously visible in the title? By the way, AI is not wrong "all the time".

Re: LLMs are steroids for your Dunning-Kruger

#66

Post by rwolf »

zkmon wrote: post #64

>> How often do you think a ChatGPT user walks away not just misinformed, but misinformed with conviction? I would bet this happens all the time. Why is the "Dunning-Kruger" is not mentioned anywhere in the article body while is gloriously visible in the title? By the way, AI is not wrong "all the time".

"all the time" in this context means frequently, not constantly, and is usually understood as a global rate.

for example: people fall in love all the time.

Re: LLMs are steroids for your Dunning-Kruger

#67

Post by Night_Thastus »

moffkalast wrote: post #41
balderdash wrote: post #16

I ascribe the effect of LLMs as similar to reading the newspaper, when I learn about something I have no knowledge base in I come away feeling like I learned a lot. When I interact with a newspaper or LLM in an area where I have real domain expertise I realize they don’t know what they are talking about - which is concerning about the information I get from them about topics I don’t have that high level of domain exp…

And why stop at newspapers, it's been a while since one could say books have any integrity, pretty much anyone can get anything into print these days. From political shenanigans to self help books designed to confirm people's biases to sell more units. Video's by far the hardest to fake but that's changing as well. Regardless of what media you get your info from you have to be selective of what sources you trust. It'…

The problem is that LLM output is so incredibly confident in tone. It really sounds like you're talking to an expert who has years of experience and has done the research for you - and tech companies push this angle quite hard.

That's bad when their output can be complete garbage at times.

Re: LLMs are steroids for your Dunning-Kruger

#68

Post by _cenw »

sho_hn wrote: post #24
Insanity wrote: post #11

Earlier quoted context omitted.

I remember back when I was in secondary school, something commonly heard was "Don't just trust wikipedia, check it's resources, because it's crowdsourced and can be wrong". Now, almost 2 decades later, I rarely hear this stance and I see people relying on wikipedia as an authoritative source of truth. i.e, linking to wikipedia instead of the underlying sources. In the same sense, I can see that "Don't trust LLMs" wil…

I don't think the cases are really the same. With Wikipedia people have learned to trust that the probability of the information being at least reasonably good is pretty high because there's an editing crucible around it and the ability to correct misinformation surgically. No one can hotpatch a LLM in 5mins.

The best LLM powered solutions are as little LLM and as much conventional search engine / semantic database lookups and handcrafted coaxing as possible. But even then, the conversational interface is nice and lets you do less handcrafting in the NLP department.

Using Perplexity or Claude in "please source your answer" mode is much more like a conventional search engine than looking up data embedded in 5 trillion (or whatever) parameters.

Re: LLMs are steroids for your Dunning-Kruger

#69

Post by cratermoon »

tayo42 wrote: post #30
Insanity wrote: post #11

Earlier quoted context omitted.

I remember back when I was in secondary school, something commonly heard was "Don't just trust wikipedia, check it's resources, because it's crowdsourced and can be wrong". Now, almost 2 decades later, I rarely hear this stance and I see people relying on wikipedia as an authoritative source of truth. i.e, linking to wikipedia instead of the underlying sources. In the same sense, I can see that "Don't trust LLMs" wil…

Wikipedia is usually close enough and most users don't require perfection for their "facts" Ive noticed things like gemini summaries on Google searches are also generally close enough.

Close enough only counts in horseshoes and hand grenades
Post reply on HN