Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are steroids for your Dunning-Kruger

bytesauna.com

Post reply on HN
Print view View original

211–220 of 308 posts

Re: LLMs are steroids for your Dunning-Kruger

#211

Post by zahlman »

ACCount37 wrote: post #191
zahlman wrote: post #186

Earlier quoted context omitted.

> What more are human brains than piles of wet meat? Calculation isn't what makes us special; that's down to things like consciousness, self-awareness and volition. > The main reason to say "LLMs are just next token predictions" is to stop thinking about all the inconvenient things. Things like... They do it by iteratively predicting the next token. Suppose the calculations to do a more detailed analysis were tractab…

Do you have, by chance, a set of benchmarks that could be administered to humans and LLMs both, and used to measure and compare the levels of "consciousness, self-awareness and volition" in them? Because if not, it's worthless philosophical drivel. If it can't be defined, let alone measured, then it might as well not exist. What is measurable and does exist: performance on specific tasks. And the pool of tasks where…

> Because if not, it's worthless philosophical drivel.

The feeling is mutual:

> ... that doesn't bode well for human intelligence being unique or exceptional in any way.

My guess was that you argued that we "don't understand" these systems, or that our incomplete analysis matters, specifically to justify the possibility that they are in whatever sense "intelligent". And now you are making that explicit.

If you think that intelligence is well-defined enough, and the definition agreed-upon enough, to argue along these lines, the sophistry is yours.

> If it can't be defined, let alone measured

In fact, we can measure things (like "intelligence") without being able to define them. We can generally agree that a person of higher IQ has been measured to be more intelligent than a person of lower IQ, even without agreeing on what was actually measured. Measurement can be indirect; we only need accept that performance on tasks on an IQ test correlates with intelligence, not necessarily that the tasks demonstrate or represent intelligence.

And similarly, based on our individual understanding of the concept of "intelligence", we may conclude that IQ test results may not be probative in specific cases, or that administering such a test is inappropriate in specific cases.

Re: LLMs are steroids for your Dunning-Kruger

#212

Post by rockostrich »

brewtide wrote: post #162
rockostrich wrote: post #128

Earlier quoted context omitted.

At the moment, I find them to be the perfect tool to get started with learning about something. I don't expect it to tell me everything I need to know or to even be right, but if I ask ChatGPT or another LLM a question about a subject I'm not familiar with then it will at least use a bunch of terminology that I didn't have in my vocabulary before starting. For example, I just bought a 1990 Miata and I want to install…

Completely not related to any LLM usage, but welcome to the world of NA Miata ownership! I think you'll find that with just general maintenance it'll treat you very well -- My '91 is the most reliable car in the drive, and by far the most whimsical. (I just got back from a Miata errand trip in the pouring rain -- Why did I drive the Miata? Winter is very soon, and it gets put away for ~3 ish months -- so at this time…

Thanks! I've been looking for a while but couldn't find one that was in decent shape for less than $10k. Thankfully for some reason people shy away from RHD cars and I snagged a 1990 Eunos Roadster for $7700 on C&B. I'm in NJ and sadly it seems like that week was the last week that would've been a decent week to drive with the top down. I may still try and take it out but I'm definitely going to be bundled up.

Re: LLMs are steroids for your Dunning-Kruger

#213

Post by knownjorbist »

kaydub wrote: post #159
rockostrich wrote: post #128

Earlier quoted context omitted.

At the moment, I find them to be the perfect tool to get started with learning about something. I don't expect it to tell me everything I need to know or to even be right, but if I ask ChatGPT or another LLM a question about a subject I'm not familiar with then it will at least use a bunch of terminology that I didn't have in my vocabulary before starting. For example, I just bought a 1990 Miata and I want to install…

Yeah, it's definitely been terrific for figuring out terminology or "the right word" to use for things.

Or to put it another way, it's great at filling in the "don't know what you don't know" gap.

Re: LLMs are steroids for your Dunning-Kruger

#214

Post by onraglanroad »

dcminter wrote: post #205
cynicalkane wrote: post #149

Earlier quoted context omitted.

I read this good breakdown on 'The Mother Tongue' on everything2 sometime ago: https://everything2.com/title/The+Mother+Tongue%253A+English...

Hmm. Why should I take this critique as being any more accurate than Bryson, given that the writer says in so many words: "[...] I - someone who’s far from an expert at linguistics [...]" The rather sniffy observation about Wikipedia falls very flat as the book was written 10 years before Wikipedia existed! In fact Bryson wrote his book a good 20 years earlier than this critique so perhaps this huffy person has resou…

> Why should I take this critique as being any more accurate than Bryson

Because you have access to various dictionaries and can easily verify it for yourself?

Assuming the quotes from the book are accurate, that's really poor.

Re: LLMs are steroids for your Dunning-Kruger

#215

Post by ACCount37 »

zahlman wrote: post #211
ACCount37 wrote: post #191

Earlier quoted context omitted.

Do you have, by chance, a set of benchmarks that could be administered to humans and LLMs both, and used to measure and compare the levels of "consciousness, self-awareness and volition" in them? Because if not, it's worthless philosophical drivel. If it can't be defined, let alone measured, then it might as well not exist. What is measurable and does exist: performance on specific tasks. And the pool of tasks where…

> Because if not, it's worthless philosophical drivel. The feeling is mutual: > ... that doesn't bode well for human intelligence being unique or exceptional in any way. My guess was that you argued that we "don't understand" these systems, or that our incomplete analysis matters, specifically to justify the possibility that they are in whatever sense "intelligent". And now you are making that explicit. If you think…

Well, you could do the funny thing, and try to measure the IQ of an LLM using human IQ tests.

Frontier models usually get somewhere between 90 and 125, including on unseen tasks. Massive error bars. The performance of frontier models keeps rising, in line with other benchmarks.

And, for all the obvious issues with the method? It's less of a worthless thing to do than claiming "LLMs don't have consciousness, self-awareness and volition, and no, not gonna give definitions, not gonna give tests, they just don't have that".

Re: LLMs are steroids for your Dunning-Kruger

#216

Post by ivape »

pklausler wrote: post #105

LLMs, kind of like Bill Bryson's books, are great at presenting "information" that seems completely plausible, authoritative, and convincing to the reader. But when you actually do know the truth about a subject, you realize how completely full of crap they too often are. And somehow after being given a patently counterfactual response to one query, we just blindly continue to take their responses to other queries as…

Can you give me examples of topics that you know about that LLMs don’t know about?

Re: LLMs are steroids for your Dunning-Kruger

#217

Post by whatevertrevor »

ACCount37 wrote: post #189
zahlman wrote: post #184

Earlier quoted context omitted.

> Saying "I know how LLMs work because I read a paper about transformer architecture" is about as delusional as saying "I read a paper about transistors, and now I understand how Ryzen 9800X3D works". Maybe more so. Which is to say, not delusional at all. Or else we have to accept that basically hardly anyone "understands" anything . You set an unrealistic standard. Beginners play abstract board games terribly. We do…

Understanding a transistor is an incredibly small part of how Ryzen 9800X3D does what it does. Is it a foundational part? Yes. But if you have it and nothing else, that adds up to knowing almost nothing about how the whole CPU works. And you could come to understand much more than that without ever learning what a "transistor" even is. Understanding low level foundations does not automatically confer the understandin…

There is so much complexity in interactions of systems that is easy to miss.

Saying that one can understand a modern CPU by understanding how a transistor works is kinda akin to saying you can understand the operation of a country by understanding a human from it. It's a necessary step, probably, but definitely not sufficient.

It also reminds me of a pet peeve in software development where it's tempting to think you understand the system from the unit tests of each component, while all the interesting stuff happens when different components interact with each other in novel ways.

Re: LLMs are steroids for your Dunning-Kruger

#218

Post by zahlman »

ACCount37 wrote: post #189
zahlman wrote: post #184

Earlier quoted context omitted.

> Saying "I know how LLMs work because I read a paper about transformer architecture" is about as delusional as saying "I read a paper about transistors, and now I understand how Ryzen 9800X3D works". Maybe more so. Which is to say, not delusional at all. Or else we have to accept that basically hardly anyone "understands" anything . You set an unrealistic standard. Beginners play abstract board games terribly. We do…

Understanding a transistor is an incredibly small part of how Ryzen 9800X3D does what it does. Is it a foundational part? Yes. But if you have it and nothing else, that adds up to knowing almost nothing about how the whole CPU works. And you could come to understand much more than that without ever learning what a "transistor" even is. Understanding low level foundations does not automatically confer the understandin…

My entire point here is that one can, in fact, reasonably claim to "understand" a system without being able to model its high level behaviors. It's not a mistake; it's disagreeing with you about what the word "understand" means.

Re: LLMs are steroids for your Dunning-Kruger

#219

Post by pklausler »

Panoramix wrote: post #163
pklausler wrote: post #105

LLMs, kind of like Bill Bryson's books, are great at presenting "information" that seems completely plausible, authoritative, and convincing to the reader. But when you actually do know the truth about a subject, you realize how completely full of crap they too often are. And somehow after being given a patently counterfactual response to one query, we just blindly continue to take their responses to other queries as…

I don't quite disagree but this comparison is typically unfair, because when you really know about a subject you tend to ask way more difficult questions than about other subjects, so of course the LLMs are gonna struggle more. If you ask really basic questions they will regurgitate well known bachelor-level knowledge and look good. What do I know about biology anyway? about silos for grain storage? any passable answ…

It's a valuable but scary experiment to query an LLM on basic subject matter in a field that you know a lot about. Ask those basic questions first.

Re: LLMs are steroids for your Dunning-Kruger

#220

Post by fennec-posix »

This article also touches on why LLMs can be so dangerous for those who are going through a psychotic episode, it will hit you with the "That's a great idea", "You're correct", etc. Which will just further play into someone's delusions, to a point it's directly down a statistical well telling the person what they want to hear. Sadly this has ended in tragedy more than a few times.
Post reply on HN