Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are steroids for your Dunning-Kruger

bytesauna.com

Post reply on HN
Print view View original

31–40 of 308 posts

Re: LLMs are steroids for your Dunning-Kruger

#31

Post by agumonkey »

sho_hn wrote: post #4

I'm not sure this is something I really worry about. Whenever I use an LLM I feel dumber, not smarter; there's a sensation of relying on a crutch instead of having done the due diligence of learning something myself. I'm less confident in the knowledge and less likely to present it as such. Is anyone really cocksure on the basis of LLM received knowledge? > As I ChatGPT user I notice that I’m often left with a sense…

Most of the time it feels like a crutch to me. There has been a few moments where it unlocked deep motivation (by having a feel for the size of a solution based on chatgpt output) and one time a research project where any crazy idea I threw, it would imagine what it would entail in terms of semantics and then I was inspired even more.

The jury is Still out on what value these things will bring

Re: LLMs are steroids for your Dunning-Kruger

#32

Post by Chabsff »

yannyu wrote: post #17
Chabsff wrote: post #3

> I feel like LLMs are a fairly boring technology. They are stochastic black boxes. The training is essentially run-of-the-mill statistical inference. There are some more recent innovations on software/hardware-level, but these are not LLM-specific really. This is pretty ironic, considering the subject matter of that blog post. It's a super-common misconception that's gained very wide popularity due to reactionary (a…

What's the misconception? LLMs are probabilistic next-token prediction based on current context, right?

Yeah, but that's their interface. That informs surprisingly little about their inner workings.

ANNs are arbitrary function approximators. The training process uses statistical methods to identify a set of parameters that approximate the function as best as possible. That doesn't necessarily mean that the end result is equivalent to a very fancy multi-stage linear regression. It's a possible outcome of the process, but it's not the only possible outcome.

Looking at a LLMs I/O structure and training process is not enough to conclude much of anything. And that's the misconception.

Re: LLMs are steroids for your Dunning-Kruger

#34

Post by aeve890 »

gopheryourshelf wrote: post #6

>“the problem with the world is that the stupid are cocksure, while the intelligent are full of doubt.” Is it me or does everyone find that dumb people seem to use this statement more than ever?

Everyone thinks they're the intelligent ones, of course. Which reinforces the repetition ad nauseam of Dunning Kruger. Which is on itself dumb AF because the effect described by Dunning and Kruger has been repeatedly exaggerated and misinterpreted. Which in turn is even dumber because Dunning-Kruger effect is debatable and reproducibility is weak at best.

Re: LLMs are steroids for your Dunning-Kruger

#35

Post by travisgriggs »

8 months or so ago, my quip regarding LLMs was “stochastic parrot.”

The term I’ve been using of late is “authority simulator.” My formative experiences with “authority figures” was a person who can speak with breadth and depth about a subject and who seems to have internalized it because they can answer quickly and thoroughly. Because LLMs do this so well, it’s really easy to feel like you’re talking to an authority in a subject. And even though my brain intellectually knows this isn’t true, emotionally, the simulation of authority is comforting.

Re: LLMs are steroids for your Dunning-Kruger

#36

Post by Brendinooo »

everdrive wrote: post #28
sho_hn wrote: post #4

I'm not sure this is something I really worry about. Whenever I use an LLM I feel dumber, not smarter; there's a sensation of relying on a crutch instead of having done the due diligence of learning something myself. I'm less confident in the knowledge and less likely to present it as such. Is anyone really cocksure on the basis of LLM received knowledge? > As I ChatGPT user I notice that I’m often left with a sense…

This captures my experience quite well. I can "get a lot more done," but it's not really me doing the things, and I feel like a bit of a fraud. And as the workday and the workweek roll on, I find myself needing to force myself to look things up and experiment rather than just asking the LLM. It's quite clear that for most people LLMs will make the more dependent. People with better discipline I think will really bene…

>it's not really me doing the things, and I feel like a bit of a fraud

I've been thinking about this a bit. We don't really think this way in other areas, is it appropriate to think this way here?

My car has an automatic transmission, am I a fraud because the machine is shifting gears for me?

My tractor plows a field, am I a fraud because I'm not using draft horses or digging manually?

Spell check caught a word, am I a fraud because I didn't look it up in a dictionary?

Re: LLMs are steroids for your Dunning-Kruger

#37

Post by kraftman »

giraffe_lady wrote: post #29
kraftman wrote: post #20

I feel like when I talk to someone and they tell me a fact, that fact goes into a kind of holding space, where I apply a filter of 'who is this person that is telling me this thing to know what the thing they are telling me is'. There's how well I know them, there's the other beleifs I know they have, there's their professional experience and their personal experience. That fact then gets marked as 'probably a true f…

The important part of this is the "I feel like" bit. There's a fair but growing bit of research that the "fact" is more durable in your memory than the context, and over time, across a lot of information, you will lose some of the mappings and integrate things you "know" to be false into model of the world. This more closely fits our models of cognition anyway. There is nothing really very like a filter in the human…

Maybe but then thats the same wether I talk to chatGPT or a human isnt it? except with chatgpt i instantly verify what im looking for, whereas with a human i cant do that.

Re: LLMs are steroids for your Dunning-Kruger

#38

Post by derektank »

aabhay wrote: post #23
Insanity wrote: post #11

Earlier quoted context omitted.

I remember back when I was in secondary school, something commonly heard was "Don't just trust wikipedia, check it's resources, because it's crowdsourced and can be wrong". Now, almost 2 decades later, I rarely hear this stance and I see people relying on wikipedia as an authoritative source of truth. i.e, linking to wikipedia instead of the underlying sources. In the same sense, I can see that "Don't trust LLMs" wil…

There’s also the fact that both Wikipedia and LLMs are non-stationary. The quality of wikipedia has grown immensely since its inception and LLMs will get more accurate (if not explicitly “smarter”)

I'm not entirely convinced that the quality of Wikipedia has improved substantially in the last decade.

Re: LLMs are steroids for your Dunning-Kruger

#39

Post by yannyu »

Chabsff wrote: post #32
yannyu wrote: post #17

Earlier quoted context omitted.

What's the misconception? LLMs are probabilistic next-token prediction based on current context, right?

Yeah, but that's their interface. That informs surprisingly little about their inner workings. ANNs are arbitrary function approximators. The training process uses statistical methods to identify a set of parameters that approximate the function as best as possible. That doesn't necessarily mean that the end result is equivalent to a very fancy multi-stage linear regression. It's a possible outcome of the process, bu…

> Yeah, but that's their interface. That informs surprisingly little about their inner workings.

I'm not sure I follow. LLMs are probabilistic next-token prediction based on current context, that is a factual, foundational statement about the technology that runs all LLMs today.

We can ascribe other things to that, such as reasoning or knowledge or agency, but that doesn't change how they work. Their fundamental architecture is well understood, even if we allow for the idea that maybe there are some emergent behaviors that we haven't described completely.

> It's a possible outcome of the process, but it's not the only possible outcome.

Again, you can ascribe these other things to it, but to say that these external descriptions of outputs call into question the architecture that runs these LLMs is a strange thing to say.

> Looking at a LLMs I/O structure and training process is not enough to conclude much of anything. And that's the misconception.

I don't see how that's a misconception. We evaluate all pretty much everything by inputs and outputs. And we use those to infer internal state. Because that's all we're capable of in the real world.

Re: LLMs are steroids for your Dunning-Kruger

#40

Post by shermantanktop »

Brendinooo wrote: post #2

>I think LLMs should not be seen as knowledge engines but as confidence engines. This is a good line, and I think it tempers the "not just misinformed, but misinformed with conviction" observation quite a bit, because sometimes moving forward with an idea at less than 100% accuracy will still bring the best outcome. Obviously that's a less than ideal thing to say, but imo (and in my experience as the former gifted st…

Confidence has multiple benefits. But one of those benefits is social - appearing confident triggers others to trust you, even when they shouldn’t.

Seeing others get burned by that pattern over and over can encourage hesitation and humility, and discourage confident action. It’s essentially an academic attitude and can be very unfortunate and self-defeating.

Post reply on HN