Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are steroids for your Dunning-Kruger

bytesauna.com

Post reply on HN
Print view View original

71–80 of 308 posts

Re: LLMs are steroids for your Dunning-Kruger

#71

Post by AlienRobot »

Night_Thastus wrote: post #67
moffkalast wrote: post #41

Earlier quoted context omitted.

And why stop at newspapers, it's been a while since one could say books have any integrity, pretty much anyone can get anything into print these days. From political shenanigans to self help books designed to confirm people's biases to sell more units. Video's by far the hardest to fake but that's changing as well. Regardless of what media you get your info from you have to be selective of what sources you trust. It'…

The problem is that LLM output is so incredibly confident in tone. It really sounds like you're talking to an expert who has years of experience and has done the research for you - and tech companies push this angle quite hard. That's bad when their output can be complete garbage at times.

It makes me really sad how Google pushes this technology that is simply flat out wrong sometimes. I forgot what exactly I searched for, but I searched for a color model that Krita supports hoping to get the online documentation as the first result and the under several Youtube thumbnails the AI overview was telling me that Krita doesn't support that color model and you need a plugin for that. Under the AI overview was the search result I was looking for about that color model in Krita.

And worse of all is that it's not even consistent, because I tried the same searches again and I couldn't get the same answer, so it just randomly decides to assert complete nonsense sometimes while other times it gives the right answer or says something completely unrelated.

It's really been a major negative in my search experience. Every time I search for something I can't be sure that it's actually quoting anything verbatim, so I need to check the sources anyway. Except it's much harder to find the link to the source with these AI's than it is to just browse the verbatim snippets in a simple list of search results. So it's just occupying space with something that is simply less convenient.

Re: LLMs are steroids for your Dunning-Kruger

#72

Post by nzach »

bryanlarsen wrote: post #55

My opinion: if LLM's speed you up, you're doing it wrong. You have to carefully review and audit every line that comes out of an LLM. You have to spend a lot of time forcing LLM's to prove that the code it wrote is correct. You should be nit-picking everything. Despite, LLM's are useful. I could write the code faster without an LLM, but then I'd have code that wasn't carefully reviewed line-by-line because my coworke…

> You have to carefully review and audit every line that comes out of an LLM. You have to spend a lot of time forcing LLM's to prove that the code it wrote is correct. You should be nit-picking everything.

I'm not sure this statement is true most of the time. This kind of reasoning reminds me of the discussion around 'code correctness'. In my opinion there are very few instances where correctness is really important. Most of the time you just need something that works well enough.

Imagine you have a continuous numeric scale that goes from 'never works' to '100% formal proofs' to indicate the correctness of every piece of software. Pushing your code to the '100% formal proofs' side takes a lot of resources, that could be deployed on other places.

Re: LLMs are steroids for your Dunning-Kruger

#73

Post by zkmon »

rwolf wrote: post #66
zkmon wrote: post #64

>> How often do you think a ChatGPT user walks away not just misinformed, but misinformed with conviction? I would bet this happens all the time. Why is the "Dunning-Kruger" is not mentioned anywhere in the article body while is gloriously visible in the title? By the way, AI is not wrong "all the time".

"all the time" in this context means frequently, not constantly, and is usually understood as a global rate. for example: people fall in love all the time.

I don't think you get "frequently" misinformed by ChatGPT. What kind of prompts you are using, that make ChatGPT to give false info very often?

Re: LLMs are steroids for your Dunning-Kruger

#76

Post by djoldman »

aabhay wrote: post #23
Insanity wrote: post #11

Earlier quoted context omitted.

I remember back when I was in secondary school, something commonly heard was "Don't just trust wikipedia, check it's resources, because it's crowdsourced and can be wrong". Now, almost 2 decades later, I rarely hear this stance and I see people relying on wikipedia as an authoritative source of truth. i.e, linking to wikipedia instead of the underlying sources. In the same sense, I can see that "Don't trust LLMs" wil…

There’s also the fact that both Wikipedia and LLMs are non-stationary. The quality of wikipedia has grown immensely since its inception and LLMs will get more accurate (if not explicitly “smarter”)

Wikipedia probably wins here because you can link to a permalink version of an article.

Re: LLMs are steroids for your Dunning-Kruger

#77

Post by vablings »

derektank wrote: post #38
aabhay wrote: post #23

Earlier quoted context omitted.

There’s also the fact that both Wikipedia and LLMs are non-stationary. The quality of wikipedia has grown immensely since its inception and LLMs will get more accurate (if not explicitly “smarter”)

I'm not entirely convinced that the quality of Wikipedia has improved substantially in the last decade.

Care to provide any counter-examples? Please make it know if you end up using Wikipedia for your source of if Wikipedia's quality has changed

Re: LLMs are steroids for your Dunning-Kruger

#78

Post by CobrastanJorji »

Insanity wrote: post #11
sho_hn wrote: post #4

I'm not sure this is something I really worry about. Whenever I use an LLM I feel dumber, not smarter; there's a sensation of relying on a crutch instead of having done the due diligence of learning something myself. I'm less confident in the knowledge and less likely to present it as such. Is anyone really cocksure on the basis of LLM received knowledge? > As I ChatGPT user I notice that I’m often left with a sense…

I remember back when I was in secondary school, something commonly heard was "Don't just trust wikipedia, check it's resources, because it's crowdsourced and can be wrong". Now, almost 2 decades later, I rarely hear this stance and I see people relying on wikipedia as an authoritative source of truth. i.e, linking to wikipedia instead of the underlying sources. In the same sense, I can see that "Don't trust LLMs" wil…

A big reason for this is that Wikipedia's source is often a book or a journal article that is either offline or behind an academic paywall. Checking the source is effectively impossible without visiting a college campus's library. The likelihood that the cited information is wrongly summarizing the contents is low enough and the cost is high enough that doing so regularly would be irrational.

Re: LLMs are steroids for your Dunning-Kruger

#79

Post by jayd16 »

djoldman wrote: post #74

Just a reminder that the "Dunning-Kruger effect" is probably not real [0]. It makes sense to refer to it as a concept but it's probably not an appropriate assumption to make about people. [0] https://www.mcgill.ca/oss/article/critical-thinking/dunning-...

This seems to move the idea that you might not understand how skilled you are to some kind of law that ties humility to knowledge more strictly.

Maybe this is my misunderstanding but I don't think the common invocation really took it as a law that the unknowledgeable always think their skills are higher.

Re: LLMs are steroids for your Dunning-Kruger

#80

Post by jayd16 »

sho_hn wrote: post #4

I'm not sure this is something I really worry about. Whenever I use an LLM I feel dumber, not smarter; there's a sensation of relying on a crutch instead of having done the due diligence of learning something myself. I'm less confident in the knowledge and less likely to present it as such. Is anyone really cocksure on the basis of LLM received knowledge? > As I ChatGPT user I notice that I’m often left with a sense…

Well I think your experience is, if not in the minority, at least not the overwhelming majority.

Lots of folks think it's amazing and greatly empowers them.

Post reply on HN