Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are steroids for your Dunning-Kruger

bytesauna.com

Post reply on HN
Print view View original

111–120 of 308 posts

Re: LLMs are steroids for your Dunning-Kruger

#111

Post by thewebguyd »

nis0s wrote: post #95

There’s a gap that LLMs are trying to fill in such cases, which is that there’s too much information that we can possibly hope to make sense of in a lifetime. Just as it’s possible to compute something incorrectly with a calculator, you can definitely be led astray by an LLM, which is why I am surprised that people think these models are good enough to replace humans at work. The only thing which makes sense is to bo…

> which is why I am surprised that people think these models are good enough to replace humans at work.

There are a lot of office jobs that I'd fit into the category of "bullshit jobs." They may serve some purpose in the huge bureaucracy of enterprises but the day to day ultimately boils doing to managing someone's calendar and sending emails.

Quite a few people at my work have now started using Copilot for their emails. It's obviously AI (at least to me), and yet, the content and formatting are an improvement over what they were sending before.

So much of the marketing hype on LLMs is about how it'll replace all the engineering work (the MBA's wet dream, to replace all the expensive labor). In reality, I think its more capable at replacing non-tech labor and middle management.

An LLM can send out an email to the team and analyze a project check-in faster, and better, than some overpaid middle manager can. I have no doubts an LLM could probably serve the role of a project management office, or a business analyst.

Sure, there should still be a human in the loop for now, but you need far, far less humans in those roles than previously.

Re: LLMs are steroids for your Dunning-Kruger

#112

Post by pessimizer »

deadbabe wrote: post #9
sho_hn wrote: post #4

I'm not sure this is something I really worry about. Whenever I use an LLM I feel dumber, not smarter; there's a sensation of relying on a crutch instead of having done the due diligence of learning something myself. I'm less confident in the knowledge and less likely to present it as such. Is anyone really cocksure on the basis of LLM received knowledge? > As I ChatGPT user I notice that I’m often left with a sense…

If you feel dumber, it’s because you’re using the LLM to do raw work instead of using it for research. It should be a google/stackoverflow replacement, not a really powerful intellisense. You should feel no dumber than using google to investigate questions.

I find that it is terrible for research, and hallucinates 25% to 90% of its references.

If you tell it to find something and give it a detailed description of what you're looking for, it will pretend like it has verified that that thing exists, and give you a bulletpoint lecture about why it is such an effective and interesting thing that 1) you didn't ask for, and 2) is really it parroting your description back to you with embellishments.

I thought I was going to be able to use LLMs primarily for research, because I have read an enormous number of things (books, papers) in my life, and I can't necessarily find them again when they would be useful. Trying to track them down through LLMs is rarely successful and always agonizing, like pulling teeth that are constantly lying to you. A surprising outcome is that I often get so frustrated by the LLM and so detailed in how I'm complaining about its stupid responses that I remind myself of something that allows me to find the reference on my own.

I have to suspect that people who find it useful for research are researching things that are easily discoverable through many other means. Those are not the things that are interesting. I totally find it useful to find something in software docs that I'm too lazy to look up myself, but it's literally saving me 10 minutes.

Re: LLMs are steroids for your Dunning-Kruger

#113

Post by hathawsh »

gowld wrote: post #96

I recently asked a leading GenAI chatbot to help me understand a certain physics concept. As I pressed it on the aspect I was confused about, the bot repeatedly explained, and in our discussion, consistently held firm that I was misunderstanding something, and made guesses about what I was misunderstanding. Eventually I realized and stated my mistake, and the chatbot confirmed and explained the difference between my…

I've seen similar results in physics. I suspect LLMs are capable of redirecting the user accurately when there have been long discussions on the web about that topic. When an LLM can pattern-match on whole discussions, it becomes a next-level search engine.

Next, I hope we can somehow get LLMs to distinguish between reliable and less-reliable results.

Re: LLMs are steroids for your Dunning-Kruger

#114

Post by ACCount37 »

svieira wrote: post #58
GMoromisato wrote: post #44

Speaking of uncertainty, I wish more people would accept their uncertainty with regards to the future of LLMs rather than dash off yet another cocksure article about how LLMs are {X}, and therefore {completely useless}|{world-changing}. Quantity has a quality of its own. The first chess engine to beat Gary Kasparov wasn't fundamentally different than earlier ones--it just had a lot more compute power. The original Go…

> How can meat think? Some of us used to think that meat spontaneously generated flies. Maybe someday we'll (re-)learn that meat doesn't spontaneously generate thought either?

I don't give much merit to ideas that demand the existence of Magic Fairy Dust.

And especially not now. Not when LLMs can already do pretty much anything that a human can - and some of those things they can even do well.

Re: LLMs are steroids for your Dunning-Kruger

#115

Post by thewebguyd »

AlienRobot wrote: post #71
Night_Thastus wrote: post #67

Earlier quoted context omitted.

The problem is that LLM output is so incredibly confident in tone. It really sounds like you're talking to an expert who has years of experience and has done the research for you - and tech companies push this angle quite hard. That's bad when their output can be complete garbage at times.

It makes me really sad how Google pushes this technology that is simply flat out wrong sometimes. I forgot what exactly I searched for, but I searched for a color model that Krita supports hoping to get the online documentation as the first result and the under several Youtube thumbnails the AI overview was telling me that Krita doesn't support that color model and you need a plugin for that. Under the AI overview wa…

The AI is also indiscriminate with what "sources" it chooses. Even deep research mode in gemini.

You can go through and look at the websites it checked, and it's 80% blogspam with no other sources cited on said blog.

When I'm manually doing a Google search, I'm not just randomly picking the first few links I'm deliberately filtering for credible domains or articles, not just picking whatever random marketing blog SEO'd their way to the top.

Sorry Gemini, an Advertorial from Times of India is not a reliable source for what I'm looking for. Nor is this xyz affiliate marketing blog stuffed to the brim with ads and product placement.

Some of that is due to that's probably 90% of the internet, but weren't these things trained on huge amounts of books, and published peer-reviewed works? Where are those in the sources?

Re: LLMs are steroids for your Dunning-Kruger

#116

Post by Ultimatt »

Insanity wrote: post #11
sho_hn wrote: post #4

I'm not sure this is something I really worry about. Whenever I use an LLM I feel dumber, not smarter; there's a sensation of relying on a crutch instead of having done the due diligence of learning something myself. I'm less confident in the knowledge and less likely to present it as such. Is anyone really cocksure on the basis of LLM received knowledge? > As I ChatGPT user I notice that I’m often left with a sense…

I remember back when I was in secondary school, something commonly heard was "Don't just trust wikipedia, check it's resources, because it's crowdsourced and can be wrong". Now, almost 2 decades later, I rarely hear this stance and I see people relying on wikipedia as an authoritative source of truth. i.e, linking to wikipedia instead of the underlying sources. In the same sense, I can see that "Don't trust LLMs" wil…

Well also years of Wikipedia proving to be more accurate than anything in print and rarely and not for very long misrepresenting source materials. For LLMs to get that same respect they would have to pull off all of the same reassuring qualities.

Re: LLMs are steroids for your Dunning-Kruger

#117

Post by yannyu »

zkmon wrote: post #73
rwolf wrote: post #66

Earlier quoted context omitted.

"all the time" in this context means frequently, not constantly, and is usually understood as a global rate. for example: people fall in love all the time.

I don't think you get "frequently" misinformed by ChatGPT. What kind of prompts you are using, that make ChatGPT to give false info very often?

It's remarkably easy to get ChatGPT to hallucinate. A go-to for me is to ask it about factual things as compared to other factual things. The description of the Bluey episode is wrong in nearly every possible way. Additionally, the response continues on analyzing in multiple ways, and again continues to get the fundamental facts and therefore the analysis wrong.

Prompt: Compare the Space episode of Bluey to Interstellar

Response: That’s a really fun and surprisingly fitting comparison! Let’s break down how Bluey’s “Space” episode parallels Christopher Nolan’s Interstellar — both emotionally and thematically — even though one is a 7-minute preschool cartoon and the other a 3-hour sci-fi epic.

1. Core Theme: Love Across Distance

Interstellar: The film centers on a father (Cooper) who leaves his daughter (Murph) to save humanity, exploring how love transcends time and space — literally. The emotional heart of Interstellar is the bond between parent and child that endures despite impossible separation.

Bluey: “Space” In this episode, Bluey, Bingo, and their dad (Bandit) play an imaginative game where they’re astronauts who have to separate to save their mission. The moment Bluey and Bandit must say goodbye echoes that same emotional tension — the pain of letting go but also the love that motivates sacrifice.

Parallel: Both stories use the setting of outer space as a metaphor for emotional distance — how people we love can feel far away, and how connection can bridge that gap.

Re: LLMs are steroids for your Dunning-Kruger

#118

Post by niccl »

pklausler wrote: post #105

LLMs, kind of like Bill Bryson's books, are great at presenting "information" that seems completely plausible, authoritative, and convincing to the reader. But when you actually do know the truth about a subject, you realize how completely full of crap they too often are. And somehow after being given a patently counterfactual response to one query, we just blindly continue to take their responses to other queries as…

> But when you actually do know the truth about a subject, you realize how completely full of crap they too often are

The Gell-Mann Amnesia Effect https://en.wikipedia.org/wiki/Gell-Mann_amnesia_effect

Re: LLMs are steroids for your Dunning-Kruger

#119

Post by catlifeonmars »

kraftman wrote: post #56
jancsika wrote: post #54

Earlier quoted context omitted.

> Afterwards I don't feel like I know something, I feel like I've got a faster broad idea of what facts might exist and where to look for them, a good set of things to investigate, etc. Can you cite a specific example where this happened for you? I'm interested in how you think you went from "broad idea" to building actual knowledge.

Sure. I wanted to tile my bathroom, from chatgpt i learned about laser levels, ledger boards, and levelling spacers (id only seen those cross corner ones before).

FWIW that seems like low stakes compared to what I see other people using LLMs for (e.g medical advice).

Re: LLMs are steroids for your Dunning-Kruger

#120

Post by serial_dev »

pklausler wrote: post #105

LLMs, kind of like Bill Bryson's books, are great at presenting "information" that seems completely plausible, authoritative, and convincing to the reader. But when you actually do know the truth about a subject, you realize how completely full of crap they too often are. And somehow after being given a patently counterfactual response to one query, we just blindly continue to take their responses to other queries as…

Similar to (same as?) Gell-Mann amnesia effect.
Post reply on HN