Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are steroids for your Dunning-Kruger

bytesauna.com

Post reply on HN
Print view View original

131–140 of 308 posts

Re: LLMs are steroids for your Dunning-Kruger

#131

Post by Swizec »

cluoma wrote: post #124

I used to get this same feeling during lectures in uni. Often the information was presented well and, along with some clear examples, everything seemed to make perfect sense. It wasn't until working through practice problems later, on my own, did it become clear how much detail I was missing.

> It wasn't until working through practice problems later, on my own, did it become clear how much detail I was missing.

This is a common problem in learning. Recognition is easier than recall and smoothness is confused for understanding.

You actually need to struggle with the concepts a bit to learn effectively. Without the struggle it feels more effective, but is not.

Re: LLMs are steroids for your Dunning-Kruger

#132

Post by nis0s »

thewebguyd wrote: post #111
nis0s wrote: post #95

There’s a gap that LLMs are trying to fill in such cases, which is that there’s too much information that we can possibly hope to make sense of in a lifetime. Just as it’s possible to compute something incorrectly with a calculator, you can definitely be led astray by an LLM, which is why I am surprised that people think these models are good enough to replace humans at work. The only thing which makes sense is to bo…

> which is why I am surprised that people think these models are good enough to replace humans at work. There are a lot of office jobs that I'd fit into the category of "bullshit jobs." They may serve some purpose in the huge bureaucracy of enterprises but the day to day ultimately boils doing to managing someone's calendar and sending emails. Quite a few people at my work have now started using Copilot for their ema…

I go back and forth on the idea that some jobs are bullshit, maybe I haven’t been exposed to enough industries or work places. Every place I worked definitely didn’t have bullshit jobs to hand out as adult daycare, but I can see how some places can become bloated because an over ambitious middle manager wants to say they manage X number of people on their resume. So there are bullshit jobs in that there are people who aren’t being utilized correctly, so in that case I’d say they’re no bullshit jobs, just bullshit leadership or managers.

Re: LLMs are steroids for your Dunning-Kruger

#133

Post by Kiro »

bryanlarsen wrote: post #108
nzach wrote: post #72

Earlier quoted context omitted.

> You have to carefully review and audit every line that comes out of an LLM. You have to spend a lot of time forcing LLM's to prove that the code it wrote is correct. You should be nit-picking everything. I'm not sure this statement is true most of the time. This kind of reasoning reminds me of the discussion around 'code correctness'. In my opinion there are very few instances where correctness is really important.…

At least for us, every bug that makes it into a release that gets installed on a client computer costs us 100x - 1000x as much as a bug that gets caught earlier.

Most code is not critical like that. A lot of the stuff I write has very little impact if things go wrong and it's easy to tell if it's incorrect.

Re: LLMs are steroids for your Dunning-Kruger

#134

Post by j-bos »

pklausler wrote: post #105

LLMs, kind of like Bill Bryson's books, are great at presenting "information" that seems completely plausible, authoritative, and convincing to the reader. But when you actually do know the truth about a subject, you realize how completely full of crap they too often are. And somehow after being given a patently counterfactual response to one query, we just blindly continue to take their responses to other queries as…

> like Bill Bryson's books, are great at presenting "information" that seems completely plausible, authoritative, and convincing to the reader. But when you actually do know the truth about a subject, you realize how completely full of crap

Wow, I have a couple Bill Bryson books on my reading list, can you share some examples of that?

Re: LLMs are steroids for your Dunning-Kruger

#135

Post by jb1991 »

pklausler wrote: post #105

LLMs, kind of like Bill Bryson's books, are great at presenting "information" that seems completely plausible, authoritative, and convincing to the reader. But when you actually do know the truth about a subject, you realize how completely full of crap they too often are. And somehow after being given a patently counterfactual response to one query, we just blindly continue to take their responses to other queries as…

I think it is truly hilarious that you brought Bill Bryson into this discussion.

Re: LLMs are steroids for your Dunning-Kruger

#136

Post by Foobar8568 »

nis0s wrote: post #95

There’s a gap that LLMs are trying to fill in such cases, which is that there’s too much information that we can possibly hope to make sense of in a lifetime. Just as it’s possible to compute something incorrectly with a calculator, you can definitely be led astray by an LLM, which is why I am surprised that people think these models are good enough to replace humans at work. The only thing which makes sense is to bo…

Spend half a day with me, and you'll understand why LLM can replace most people in a company.

Re: LLMs are steroids for your Dunning-Kruger

#137

Post by gspencley »

miningape wrote: post #52
Insanity wrote: post #11

Earlier quoted context omitted.

I remember back when I was in secondary school, something commonly heard was "Don't just trust wikipedia, check it's resources, because it's crowdsourced and can be wrong". Now, almost 2 decades later, I rarely hear this stance and I see people relying on wikipedia as an authoritative source of truth. i.e, linking to wikipedia instead of the underlying sources. In the same sense, I can see that "Don't trust LLMs" wil…

> "Don't just trust wikipedia, check it's resources, because it's crowdsourced and can be wrong" This comes from decades of teachers misremembering what the rule was, and eventually it morphed into the Wikipedia specific form we see today - the actual rule is that you cannot cite an encyclopaedia in an academic paper. full stop. Wikipedia is an encyclopaedia and therefore should not be cited. Wikipedia is the only en…

More colloquially, people would say that Wikipedia could not be trusted because "anyone can edit the pages or write whatever they want."

Of course that's demonstrative of the genesis fallacy. Anyone can write or publish a book, too. So it always comes down to "how can you trust information?" That's where individual responsibility to think critically comes in. There's not really anything you can do about the fact that a lot of people will choose to not think.

Re: LLMs are steroids for your Dunning-Kruger

#138

Post by ACCount37 »

parineum wrote: post #51
Chabsff wrote: post #3

> I feel like LLMs are a fairly boring technology. They are stochastic black boxes. The training is essentially run-of-the-mill statistical inference. There are some more recent innovations on software/hardware-level, but these are not LLM-specific really. This is pretty ironic, considering the subject matter of that blog post. It's a super-common misconception that's gained very wide popularity due to reactionary (a…

I'm not sure what claim your disputing or making with this. What more are LLMs than statistical inference machines? I don't know that I'd assert that's all they are with confidence but all the configurations options I can play with during generation (Top K, Top P, Temperature, etc.) are all ways to _not_ select the most likely next token which leads me to believe that they are, in fact, just statistical inference mac…

What more are human brains than piles of wet meat?

It's not an argument - it's a dismissal. It's boneheaded refusal to think on the matter in any depth, or consider any of the implications.

The main reason to say "LLMs are just next token predictions" is to stop thinking about all the inconvenient things. Things like "how the fuck does training on piles of text make machines that can write new short stories" or "why is a big fat pile of matrix multiplications better at solving unseen math problems than I am".

Re: LLMs are steroids for your Dunning-Kruger

#139

Post by dfxm12 »

jumploops wrote: post #127

I recall trying to use GPT-4 to plan a trip through the PNW in ~Spring of 2023. It presented a reasonable agenda, however 80% of the rockhounding spots were completely made up! Over time, and as LLMs have gotten less sycophantic, I’ve found myself trusting them a bit more (a dangerous and slippery slope). With that said, GPT-4o in particular, seemed to rank user satisfaction above truth. I’ve found that GPT-5 Pro is…

As always, trust, but verify! Google maps lists "made up" places or outdated info. AI isn't scouting these locations physically...

Of course, at that point, the real question is, what's the value difference (taking into account personal, external and social costs) between asking chatgpt and /r/rockhounding (or whatever message boards they frequent)? At least if you start a thread on reddit, you might meet other people in the area with the same hobby, find a spot no one's talked about yet, get expert context and leave a trail for others to find.

Re: LLMs are steroids for your Dunning-Kruger

#140

Post by withinboredom »

daotoad wrote: post #109
hn_acc1 wrote: post #92

Earlier quoted context omitted.

Except when they glaringly get things wrong like "character X on show Y said catchphrase Z", and two queries produce two different values of X, one right, one wrong. The more I use gemini summaries for things I know a bit about, the worse my opinion of them..

https://en.wikipedia.org/wiki/Gell-Mann_amnesia_effect

Thanks for the Wikipedia link, do you have a source? /s
Post reply on HN