Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are steroids for your Dunning-Kruger

bytesauna.com

Post reply on HN
Print view View original

101–110 of 308 posts

Re: LLMs are steroids for your Dunning-Kruger

#101

Post by thewebguyd »

Insanity wrote: post #11
sho_hn wrote: post #4

I'm not sure this is something I really worry about. Whenever I use an LLM I feel dumber, not smarter; there's a sensation of relying on a crutch instead of having done the due diligence of learning something myself. I'm less confident in the knowledge and less likely to present it as such. Is anyone really cocksure on the basis of LLM received knowledge? > As I ChatGPT user I notice that I’m often left with a sense…

I remember back when I was in secondary school, something commonly heard was "Don't just trust wikipedia, check it's resources, because it's crowdsourced and can be wrong". Now, almost 2 decades later, I rarely hear this stance and I see people relying on wikipedia as an authoritative source of truth. i.e, linking to wikipedia instead of the underlying sources. In the same sense, I can see that "Don't trust LLMs" wil…

> I can see that "Don't trust LLMs" will slowly fade away and people will blindly trust them.

That's already happening. I don't even think we had a very long "Don't trust LLMs" phase, if we did it was very short.

The "normies" already trust whatever they spit out. At leadership meetings at my work, if I say anything that goes against the marketing hype for LLMs, such as talking about "Don't trust LLMs", it's met with eye rolls and I'm not forward thinking enough, blah blah.

Management-types have 100% bought into the hype and are increasingly more difficult to convince otherwise.

Re: LLMs are steroids for your Dunning-Kruger

#102

Post by jermaustin1 »

miningape wrote: post #52
Insanity wrote: post #11

Earlier quoted context omitted.

I remember back when I was in secondary school, something commonly heard was "Don't just trust wikipedia, check it's resources, because it's crowdsourced and can be wrong". Now, almost 2 decades later, I rarely hear this stance and I see people relying on wikipedia as an authoritative source of truth. i.e, linking to wikipedia instead of the underlying sources. In the same sense, I can see that "Don't trust LLMs" wil…

> "Don't just trust wikipedia, check it's resources, because it's crowdsourced and can be wrong" This comes from decades of teachers misremembering what the rule was, and eventually it morphed into the Wikipedia specific form we see today - the actual rule is that you cannot cite an encyclopaedia in an academic paper. full stop. Wikipedia is an encyclopaedia and therefore should not be cited. Wikipedia is the only en…

I remember all of our encyclopedias being decades out of date growing up. My parents bought a set of Encyclopedia Brittanica in 1976 or something like that, so by the time I was reading the Encyclopedia for research on papers in the late 90s and early 00s, it was without a doubt less factual than even the earliest incarnation of Wikipedia was.

Either way, you are correct, we weren't allowed to cite any encyclopedia, but they were meant to be jumping off points for papers. After Wikipedia launched when I was in 9th grade, we weren't allowed to even look at it (blocked from school computers).

I definitely used it though.

Re: LLMs are steroids for your Dunning-Kruger

#103

Post by yannyu »

kmijyiyxfbklao wrote: post #81
yannyu wrote: post #39

Earlier quoted context omitted.

> Yeah, but that's their interface. That informs surprisingly little about their inner workings. I'm not sure I follow. LLMs are probabilistic next-token prediction based on current context, that is a factual, foundational statement about the technology that runs all LLMs today. We can ascribe other things to that, such as reasoning or knowledge or agency, but that doesn't change how they work. Their fundamental arch…

Then why not say "they are just computer programs"? I think the reason people don't say that is because they want to say "I already understand what they are, and I'm not impressed and it's nothing new". But what the comment you are replying to is saying is that the inner workings are the important innovative stuff.

> Then why not say "they are just computer programs"?

LLMs are probabilistic or non-deterministic computer programs, plenty of people say this. That is not much different than saying "LLMs are probabilistic next-token prediction based on current context".

> I think the reason people don't say that is because they want to say "I already understand what they are, and I'm not impressed and it's nothing new". But what the comment you are replying to is saying is that the inner workings are the important innovative stuff.

But we already know the inner workings. It's transformers, embeddings, and math at a scale that we couldn't do before 2015. We already had multi-layer perceptrons with backpropagation and recurrent neural networks and markov chains before this, but the hardware to do this kind of contextual next-token prediction simply didn't exist at those times.

I understand that it feels like there's a lot going on with these chatbots, but half of the illusion of chatbots isn't even the LLM, it's the context management that is exceptionally mundane compared to the LLM itself. These things are combined with a carefully crafted UX to deliberately convey the impression that you're talking to a human. But in the end, it is just a program and it's just doing context management and token prediction that happens to align (most of the time) with human expectations because it was designed to do so.

The two of you seem to be implying there's something spooky or mysterious happening with LLMs that goes beyond our comprehension of them, but I'm not seeing the components of your argument for this.

Re: LLMs are steroids for your Dunning-Kruger

#104

Post by AaronAPU »

LLMs basically act as defense attorneys for all your dumbest ideas. It is very easy to assume their confidence in you is justified, especially if you already lean narcissistic.

You now see threads on X of famous people using Grok to explain how smart their ideas are. But there’s a problem: You can literally get it to do that with every single dumb idea.

Re: LLMs are steroids for your Dunning-Kruger

#105

Post by pklausler »

LLMs, kind of like Bill Bryson's books, are great at presenting "information" that seems completely plausible, authoritative, and convincing to the reader. But when you actually do know the truth about a subject, you realize how completely full of crap they too often are. And somehow after being given a patently counterfactual response to one query, we just blindly continue to take their responses to other queries as having value.

Re: LLMs are steroids for your Dunning-Kruger

#106

Post by ecocentrik »

ok_dad wrote: post #89
miningape wrote: post #52

Earlier quoted context omitted.

> "Don't just trust wikipedia, check it's resources, because it's crowdsourced and can be wrong" This comes from decades of teachers misremembering what the rule was, and eventually it morphed into the Wikipedia specific form we see today - the actual rule is that you cannot cite an encyclopaedia in an academic paper. full stop. Wikipedia is an encyclopaedia and therefore should not be cited. Wikipedia is the only en…

The original rule when I was a lad (when wikipedia was a baby) was, "don't trust stuff on the internet, especially Wikipedia where people can change it at will." Today they might have better trust for Wikipedia-- and I know I use it as a source of truth for a lot of things-- but back in my day teachers were of the opinion that it couldn't be trusted. This was for like middle and high school, not college or university…

There was a period of time where Wikipedia was more scrutinized than print encyclopedias because people did not understand the power of having 1000s of experts and the occasional non-experts editing an entry for free instead of underpaying one sudo-expert. They couldn't comprehend how an open source encyclopedia would even work or trust that humans could effectively collaborate on the task. They imagined that 1000s of self-interested chaos monkeys would spend all of their energy destroying what 2-3 hard working people has spent hours creating instead of the inverse. Humans are very pessimistic about other humans. In my experience when humans are given the choice to cooperate or fight, most choose to cooperate.

All of that said, I trust Wikipedia more than I trust any LLMs but don't rely on either as a final source for understanding complex topics.

Re: LLMs are steroids for your Dunning-Kruger

#107

Post by thewebguyd »

Brendinooo wrote: post #36
everdrive wrote: post #28

Earlier quoted context omitted.

This captures my experience quite well. I can "get a lot more done," but it's not really me doing the things, and I feel like a bit of a fraud. And as the workday and the workweek roll on, I find myself needing to force myself to look things up and experiment rather than just asking the LLM. It's quite clear that for most people LLMs will make the more dependent. People with better discipline I think will really bene…

>it's not really me doing the things, and I feel like a bit of a fraud I've been thinking about this a bit. We don't really think this way in other areas, is it appropriate to think this way here? My car has an automatic transmission, am I a fraud because the machine is shifting gears for me? My tractor plows a field, am I a fraud because I'm not using draft horses or digging manually? Spell check caught a word, am I…

I mostly agree with this. LLMs are just another tool, and we've learned how to use and adapted to using many other tools throughout our history just fine.

With the caveat of for our field in particular, it's one of the few that require continuous learning and adaptation, so tech workers in a way are better predisposed to this line of thinking and tool adoption without some of the potential harmful side effects.

To pick on spell check, it has been showing that we can develop a dependency on it and thereby losing our own ability to spell and reason about language. But, is that a bad thing? I don't know.

What I do know is humans have been outsourcing our thinking for a long time. LLMs are another evolution in that process, just another way to push off cognitive load onto a tool like we've done with stone tablets, books, paper notes, digital notes, google, etc.

Re: LLMs are steroids for your Dunning-Kruger

#108

Post by bryanlarsen »

nzach wrote: post #72
bryanlarsen wrote: post #55

My opinion: if LLM's speed you up, you're doing it wrong. You have to carefully review and audit every line that comes out of an LLM. You have to spend a lot of time forcing LLM's to prove that the code it wrote is correct. You should be nit-picking everything. Despite, LLM's are useful. I could write the code faster without an LLM, but then I'd have code that wasn't carefully reviewed line-by-line because my coworke…

> You have to carefully review and audit every line that comes out of an LLM. You have to spend a lot of time forcing LLM's to prove that the code it wrote is correct. You should be nit-picking everything. I'm not sure this statement is true most of the time. This kind of reasoning reminds me of the discussion around 'code correctness'. In my opinion there are very few instances where correctness is really important.…

At least for us, every bug that makes it into a release that gets installed on a client computer costs us 100x - 1000x as much as a bug that gets caught earlier.

Re: LLMs are steroids for your Dunning-Kruger

#109

Post by daotoad »

hn_acc1 wrote: post #92
tayo42 wrote: post #30

Earlier quoted context omitted.

Wikipedia is usually close enough and most users don't require perfection for their "facts" Ive noticed things like gemini summaries on Google searches are also generally close enough.

Except when they glaringly get things wrong like "character X on show Y said catchphrase Z", and two queries produce two different values of X, one right, one wrong. The more I use gemini summaries for things I know a bit about, the worse my opinion of them..

https://en.wikipedia.org/wiki/Gell-Mann_amnesia_effect

Re: LLMs are steroids for your Dunning-Kruger

#110

Post by bob1029 »

I find the biggest crime with LLMs to be the size of the problems we feed them.

Every time I start getting lazy and asking ChatGPT things like "write me a singleton that tracks progression for XYZ in a unity project", I wind up with a big hole where some deeper understanding of my problem should be. A better approach is to prompt it like "Show me a few ways to persist progression-like data in a unity project. Compare and contrast them".

Having an LLM development policy where you ~blindly accept a solution simply because it works is like an HOV lane to hell. It is very tempting to do this when you are tired or in a rush. I do it all the time.

Post reply on HN