Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are steroids for your Dunning-Kruger

bytesauna.com

Post reply on HN
Print view View original

191–200 of 308 posts

Re: LLMs are steroids for your Dunning-Kruger

#191

Post by ACCount37 »

zahlman wrote: post #186
ACCount37 wrote: post #138

Earlier quoted context omitted.

What more are human brains than piles of wet meat? It's not an argument - it's a dismissal. It's boneheaded refusal to think on the matter in any depth, or consider any of the implications. The main reason to say "LLMs are just next token predictions" is to stop thinking about all the inconvenient things. Things like "how the fuck does training on piles of text make machines that can write new short stories" or "why…

> What more are human brains than piles of wet meat? Calculation isn't what makes us special; that's down to things like consciousness, self-awareness and volition. > The main reason to say "LLMs are just next token predictions" is to stop thinking about all the inconvenient things. Things like... They do it by iteratively predicting the next token. Suppose the calculations to do a more detailed analysis were tractab…

Do you have, by chance, a set of benchmarks that could be administered to humans and LLMs both, and used to measure and compare the levels of "consciousness, self-awareness and volition" in them?

Because if not, it's worthless philosophical drivel. If it can't be defined, let alone measured, then it might as well not exist.

What is measurable and does exist: performance on specific tasks.

And the pool of tasks where humans confidently outperform LLMs is both finite and ever diminishing. That doesn't bode well for human intelligence being unique or exceptional in any way.

Re: LLMs are steroids for your Dunning-Kruger

#192

Post by adamhartenz »

galaxyLogic wrote: post #188

> LLMs should not be seen as knowledge engines but as confidence engines. The thing I like best about LLM is when I ask question about some technical problem, and it tells that it is a KNOWN problem. It thus gives me confiidence that I don't need to spend time) to look for solution where there is no good soloution. Just go around it somehow. It let's me know I'm not the only person with this problem. And that way it…

This is the kind of stuff AI lies about all the time. I can get it to tell me "That is some good insight, and is a known issue..." with things I make up out of thin air.

Re: LLMs are steroids for your Dunning-Kruger

#193

Post by izzydata »

marknutter wrote: post #180
goalieca wrote: post #177

Earlier quoted context omitted.

Now let me ask you the more fundamental question.. did this do you any better than if you had searched a youtube video or some other source? Would this be video from 2016 be relevant? This may not be the right video but my approach for DIY in the last 10-20 years was to hit youtube up. https://www.youtube.com/watch?v=77q9KtjnNTU I'm trying to gauge whether LLMs are truly expanding our capabilities in a fundamental wa…

For obscure things, it's often very hard to find videos like that, and the videos vary greatly in quality. ChatGPT helped me fix my washing machine and my dryer yesterday with perfect advice, walking me through every step. Those are both projects I would've made a half assed attempt at and then thrown my hands up and called someone to do in the past.

I wonder if that can be attributed to search engines and search fields on various websites being intentionally worsened in order to push specific content and ads.

Google search and Youtube search used to almost always get you what you were looking for. Now you have to fight with it to maybe get what you are looking for because of all the sponsored ads.

Search used to be a nearly solved problem.

Re: LLMs are steroids for your Dunning-Kruger

#194

Post by beng-nl »

pklausler wrote: post #105

LLMs, kind of like Bill Bryson's books, are great at presenting "information" that seems completely plausible, authoritative, and convincing to the reader. But when you actually do know the truth about a subject, you realize how completely full of crap they too often are. And somehow after being given a patently counterfactual response to one query, we just blindly continue to take their responses to other queries as…

geLLMan amnesia

Re: LLMs are steroids for your Dunning-Kruger

#195

Post by inshard »

The author misses the science of emergence. Reductionist views can’t fully explain macro-level capabilities that arise in these systems. Something emerges at higher scales from the possibility space as model sizes grow; they stop being mere “stochastic parrots” or black boxes running simple regressions.

The weights develop their own inherent logic based on how they relate to each other, analogous to how brain waves encode memory at a level higher than individual neuron networks.

Ultimately, the value of AI lies in the imagination of its wielder. The Unknown Unknowns framework is a useful tool for navigating AI effectively (it powerful to help elaborate on Known Unknowns and identify Unknown Unknowns), along with a healthy dose of critical thinking and understanding how reinforcement learning and RLHF work post-pretraining.

Re: LLMs are steroids for your Dunning-Kruger

#196

Post by otabdeveloper4 »

habibur wrote: post #161

I think it's ok. When wikipedia arrived, everyone was up in arms that people are learning from something that's open for anyone to edit. But it rectified itself. The same thing happened when Internet arrived. "Don't believe anything you read on the Internet." I guess the reaction was same when printed media arrived. But the thing is, things get better over time.

> But it rectified itself.

Or did it?

Re: LLMs are steroids for your Dunning-Kruger

#197

Post by Schiendelman »

bryanlarsen wrote: post #108
nzach wrote: post #72

Earlier quoted context omitted.

> You have to carefully review and audit every line that comes out of an LLM. You have to spend a lot of time forcing LLM's to prove that the code it wrote is correct. You should be nit-picking everything. I'm not sure this statement is true most of the time. This kind of reasoning reminds me of the discussion around 'code correctness'. In my opinion there are very few instances where correctness is really important.…

At least for us, every bug that makes it into a release that gets installed on a client computer costs us 100x - 1000x as much as a bug that gets caught earlier.

Cost to fix, yes.

Sometimes getting the new capability around that bug to market faster is worth the tradeoff, because the revenue or market position from the capability with that bug is way more important to the business than the 1000x cost of the fix after distribution.

Re: LLMs are steroids for your Dunning-Kruger

#198

Post by flashfaffe2 »

Without being to self-centred here but since I have been using LLM heavily I have always challenged the results given

The post seems to propose the following vector:

Idea-> LLm validation -> confidence -> no further checks

My process is more :

Idea-> LLm response -> skeptical reflection -> adversarial prompting -> synthesis

Re: LLMs are steroids for your Dunning-Kruger

#199

Post by a_t48 »

brewtide wrote: post #162
rockostrich wrote: post #128

Earlier quoted context omitted.

At the moment, I find them to be the perfect tool to get started with learning about something. I don't expect it to tell me everything I need to know or to even be right, but if I ask ChatGPT or another LLM a question about a subject I'm not familiar with then it will at least use a bunch of terminology that I didn't have in my vocabulary before starting. For example, I just bought a 1990 Miata and I want to install…

Completely not related to any LLM usage, but welcome to the world of NA Miata ownership! I think you'll find that with just general maintenance it'll treat you very well -- My '91 is the most reliable car in the drive, and by far the most whimsical. (I just got back from a Miata errand trip in the pouring rain -- Why did I drive the Miata? Winter is very soon, and it gets put away for ~3 ish months -- so at this time…

This weekend I stumbled upon a cars and coffee in Fremont. Was expecting a wide variety of cars, and was surprised to see instead all Miatas.

Re: LLMs are steroids for your Dunning-Kruger

#200

Post by dcotter »

everdrive wrote: post #42
Brendinooo wrote: post #36

Earlier quoted context omitted.

>it's not really me doing the things, and I feel like a bit of a fraud I've been thinking about this a bit. We don't really think this way in other areas, is it appropriate to think this way here? My car has an automatic transmission, am I a fraud because the machine is shifting gears for me? My tractor plows a field, am I a fraud because I'm not using draft horses or digging manually? Spell check caught a word, am I…

I've been thinking about that comparison as well. A common fantasy is that civilization will collapse and the guy who knows how to hunt and start a fire will really excel. In practice, this never happens and he's sort of left behind unless he also has other skills relevant to the modern world. And, for instance, I have barely any knowledge of how my computer works, but it's a tool I use to do my job. (and to have fun…

> But for LLMs, my task might be something like "setting up apache is easy, but I've never done it so just tell me how do it so I don't fumble through learning and make it take way longer." The task was setting up Apache. The task was assigned to me, but I didn't really do it. There wasn't necessarily some higher level task that I merely needed Apache for. Apache was the whole task! And I didn't do it!

To play devil's advocate: Setting up Apache was your task. A) Either it was a one-off that you'll never have to do again, in which case it wasn't very important that you learn the process inside and out, or b) it is a task you'll have to do again (and again), and having the LLM walk you through the setup the first time acts as training wheels (unless you just lazily copy & paste and let it become a crutch).

I frequently have the LLM walk me through an unfamiliar task and, depending on several factors such as whether I expect to have to do it again soon, the urgency of the task, and my interest and/or energy at the moment, I will ask the LLM follow-up questions, challenge it on far-fetched claims, investigate alternative techniques, etc. Execute one command at a time, once you've understood what it's meant to do, what the program you're running does, how its parameters change what it does, and so on, and let the LLM help you get the picture.

The alternative is to try to piece together a complete picture of the process from official documentation like tutorials & user manuals, disparate bits of information in search results, possibly wrong and/or incomplete information from Q&A forums, and muddle through lots of trial and error. Time-consuming, labor-intensive, and much less efficient at giving your a broad-strokes idea of how the whole thing works.

I much prefer the back-and-forth with the LLM and think it gives me a better understanding of the big picture than the slow and frustrating muddling approach.

Post reply on HN