Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are steroids for your Dunning-Kruger

bytesauna.com

Post reply on HN
Print view View original

201–210 of 308 posts

Re: LLMs are steroids for your Dunning-Kruger

#201

Post by topaz0 »

vehemenz wrote: post #8

I hate to comment on just a headline—thought I did read the article—but it's wrong enough to warrant correcting. This is not what the Dunning-Kruger effect is. It's lacking metacognitive ability to understand one's own skill level. Overconfidence resulting from ignorance isn't the same thing. Joe Rogan propagated the version of this phenomenon that infiltrated public consciousness, and we've been stuck with it ever s…

Funnily enough, DK is also not real -- just a statistical artifact of a poorly chosen analysis.

Re: LLMs are steroids for your Dunning-Kruger

#202

Post by pedalpete »

I quite regularly ask LLMs to take the other side of an argument, or to tell me where something is wrong.

Unfortunately, they don't seem very good at this process, and in some ways seem to defend the previous position.

Does anyone else take this approach and have success with it?

Re: LLMs are steroids for your Dunning-Kruger

#203

Post by avs733 »

cluoma wrote: post #124

I used to get this same feeling during lectures in uni. Often the information was presented well and, along with some clear examples, everything seemed to make perfect sense. It wasn't until working through practice problems later, on my own, did it become clear how much detail I was missing.

you are not the only one. There was a paper covering this exact topic in the Proceedings of the National Academy of Sciences a few years back [0].

Passive learning (lecture) scored better on:

* Student Enjoyment

* Feeling of Learning

* Instructor effectiveness

* I wish all my courses where taught this way

Active Learning (i.e., not lecture) scored better on:

* Actual learning

The differences are not small.

[0] https://www.pnas.org/doi/10.1073/pnas.1821936116

Re: LLMs are steroids for your Dunning-Kruger

#204

Post by throwaway198846 »

smw wrote: post #126
gowld wrote: post #90

Earlier quoted context omitted.

Both comments are missing the reason that an encyclopedia should not be cited: An encyclopedia does not cite its sources, and does not claim to be a primary source, its potentialymistakes cannot be checked. (Wikipedia has the additional problem that, by default, the version cited is the ever-changing "latest" version, not a fixed and identified version.)

Maybe you haven't used Wikipedia? It very definitely cites its sources. Material that doesn't have a cited source is removed regularly.

There is plenty of not properly sourced claims on Wikipedia

Re: LLMs are steroids for your Dunning-Kruger

#205

Post by dcminter »

cynicalkane wrote: post #149
j-bos wrote: post #134

Earlier quoted context omitted.

> like Bill Bryson's books, are great at presenting "information" that seems completely plausible, authoritative, and convincing to the reader. But when you actually do know the truth about a subject, you realize how completely full of crap Wow, I have a couple Bill Bryson books on my reading list, can you share some examples of that?

I read this good breakdown on 'The Mother Tongue' on everything2 sometime ago: https://everything2.com/title/The+Mother+Tongue%253A+English...

Hmm. Why should I take this critique as being any more accurate than Bryson, given that the writer says in so many words:

"[...] I - someone who’s far from an expert at linguistics [...]"

The rather sniffy observation about Wikipedia falls very flat as the book was written 10 years before Wikipedia existed!

In fact Bryson wrote his book a good 20 years earlier than this critique so perhaps this huffy person has resources to draw upon that were not available in 1990.

Not that I really expect Bryson's stuff to dot every i and cross every t - he's a humourist.

Re: LLMs are steroids for your Dunning-Kruger

#206

Post by mwcampbell »

everdrive wrote: post #42
Brendinooo wrote: post #36

Earlier quoted context omitted.

>it's not really me doing the things, and I feel like a bit of a fraud I've been thinking about this a bit. We don't really think this way in other areas, is it appropriate to think this way here? My car has an automatic transmission, am I a fraud because the machine is shifting gears for me? My tractor plows a field, am I a fraud because I'm not using draft horses or digging manually? Spell check caught a word, am I…

I've been thinking about that comparison as well. A common fantasy is that civilization will collapse and the guy who knows how to hunt and start a fire will really excel. In practice, this never happens and he's sort of left behind unless he also has other skills relevant to the modern world. And, for instance, I have barely any knowledge of how my computer works, but it's a tool I use to do my job. (and to have fun…

> the distinction is whether or not something enables me to perform a task, or whether it's just doing the task for me.

I think school has taught us to believe that if we're assigned a task, and we take a shortcut to avoid doing the task ourselves, that's wrong. And yes, when the purpose is to learn the task or the underlying concepts, that's probably true. But in a job environment, the employer presumably only cares that the task got done in the most efficient way possible.

Edit to add: When configuring or using a particular program is tedious and/or difficult enough that you feel the need to turn to an LLM for help, I think it's an indication that a better program is needed. Having an LLM configure or operate a computer program for you is kind of like having a robot operate a computer UI that was designed for humans, as opposed to having a higher-level program just do the higher-level automation directly. In the specific case of the Apache HTTP Server, depending on what you need to do, you may find that Caddy is easy enough that you can configure it yourself without requiring the LLM. For common web server scenarios, a Caddyfile is very short, much shorter than a typical Apache or nginx configuration.

Re: LLMs are steroids for your Dunning-Kruger

#207

Post by oblio »

goalieca wrote: post #177
rockostrich wrote: post #128

Earlier quoted context omitted.

At the moment, I find them to be the perfect tool to get started with learning about something. I don't expect it to tell me everything I need to know or to even be right, but if I ask ChatGPT or another LLM a question about a subject I'm not familiar with then it will at least use a bunch of terminology that I didn't have in my vocabulary before starting. For example, I just bought a 1990 Miata and I want to install…

Now let me ask you the more fundamental question.. did this do you any better than if you had searched a youtube video or some other source? Would this be video from 2016 be relevant? This may not be the right video but my approach for DIY in the last 10-20 years was to hit youtube up. https://www.youtube.com/watch?v=77q9KtjnNTU I'm trying to gauge whether LLMs are truly expanding our capabilities in a fundamental wa…

They're a slightly better search, because web search has degraded. They also provide needed vocabulary almost directly, which accelerates search.

I would say, for big decisions (financial, work projects, health, etc), you really need the sources and you need to double check things, but I would say that maybe 70% of my searches are closer to trivia than to life changing things, so LLMs are obviously very good for that. And frequently the stuff I search is trivially verifiable, so that's also good.

The bigger worry is that the general public doesn't have the mental immune system to actually know what to look for and especially to validate the LLM answers, so we're in for a world of hurt.

We will soon have some extremely brainwashed individuals.

Re: LLMs are steroids for your Dunning-Kruger

#208

Post by tstrimple »

goalieca wrote: post #177
rockostrich wrote: post #128

Earlier quoted context omitted.

At the moment, I find them to be the perfect tool to get started with learning about something. I don't expect it to tell me everything I need to know or to even be right, but if I ask ChatGPT or another LLM a question about a subject I'm not familiar with then it will at least use a bunch of terminology that I didn't have in my vocabulary before starting. For example, I just bought a 1990 Miata and I want to install…

Now let me ask you the more fundamental question.. did this do you any better than if you had searched a youtube video or some other source? Would this be video from 2016 be relevant? This may not be the right video but my approach for DIY in the last 10-20 years was to hit youtube up. https://www.youtube.com/watch?v=77q9KtjnNTU I'm trying to gauge whether LLMs are truly expanding our capabilities in a fundamental wa…

One challenging thing about searching in new domains is you don't necessarily have the vocabulary to adequately ask the right question or use the right terms to unlock the secret knowledge. If I type my dryer symptoms into an LLM and it tells me that the drum rollers are likely bad and need to be replaced, I can take that information to Youtube or Google and get more targeted advice. The LLM can also, and often does ask leading questions to help narrow down the list of possible options.

Re: LLMs are steroids for your Dunning-Kruger

#209

Post by rockostrich »

goalieca wrote: post #177
rockostrich wrote: post #128

Earlier quoted context omitted.

At the moment, I find them to be the perfect tool to get started with learning about something. I don't expect it to tell me everything I need to know or to even be right, but if I ask ChatGPT or another LLM a question about a subject I'm not familiar with then it will at least use a bunch of terminology that I didn't have in my vocabulary before starting. For example, I just bought a 1990 Miata and I want to install…

Now let me ask you the more fundamental question.. did this do you any better than if you had searched a youtube video or some other source? Would this be video from 2016 be relevant? This may not be the right video but my approach for DIY in the last 10-20 years was to hit youtube up. https://www.youtube.com/watch?v=77q9KtjnNTU I'm trying to gauge whether LLMs are truly expanding our capabilities in a fundamental wa…

> did this do you any better than if you had searched a youtube video or some other source?

Yes, because when I searched youtube for "miata wink mod" almost all of the results were for kits for microcontrollers which I wanted to avoid because I just want to control the motors with switches. Now I know to include "SPDT" in my search and I can find more targeted videos that add an override using switches.

The video you linked is relevant but doesn't really match what I want to do. The NA Miata has a motor for each pop-up headlight. There's a dedicated button that controls the headlights popping up and down but the light switch on the turn signal overrides this if the lights are on i.e. the relay is DPDT that's an OR of the 2 signals.

I want to add rocker switch for each light where the signal from the rocker switch overrides the behavior from the existing relay. If a given DPDT rocker switch is in neutral then the signal from the relay is used but if the rocker switch is engaged in either direction then the motor moves in that direction. ChatGPT did explain a lot about the default behavior and included a lot of the terminology that helped me confirm that. Of course, if I already knew about relays then I wouldn't have needed any of this, but I didn't.

Re: LLMs are steroids for your Dunning-Kruger

#210

Post by [deleted] »

dcminter wrote: post #205
cynicalkane wrote: post #149

Earlier quoted context omitted.

I read this good breakdown on 'The Mother Tongue' on everything2 sometime ago: https://everything2.com/title/The+Mother+Tongue%253A+English...

Hmm. Why should I take this critique as being any more accurate than Bryson, given that the writer says in so many words: "[...] I - someone who’s far from an expert at linguistics [...]" The rather sniffy observation about Wikipedia falls very flat as the book was written 10 years before Wikipedia existed! In fact Bryson wrote his book a good 20 years earlier than this critique so perhaps this huffy person has resou…

[deleted]
Post reply on HN