The same way that I handle anyone who blindly trusts anything on the internet. Could be an LLM, TikTok or YouTube video, Wikipedia article, news article, whatever.
It usually involves some form of "well, no, hold on..."
Ask HN: How do you deal with people who trust LLMs?
41–50 of 236 posts
Re: Ask HN: How do you deal with people who trust LLMs?
#42Ask them to tell the LLM it's wrong... then when it goes "You are absolutely right!" to challenge it and say that it was a test. Then when it replies, ask it if it's 100% sure. They'll lose faith pretty quick.
This is an oft-repeated meme, but I’m convinced the people saying it are either blindly repeating it, using bad models/system prompts, or some other issue. Claude Opus will absolutely push back if you disagree. I routinely push back on Claude only to discover on further evaluation that the model was correct.
As a test I just did exactly what you said in a Claude Opus 4.6 session about another HN thread. Claude considered* the contradiction, evaluated additional sources, and responded backing up its original claim with more evidence.
I will add that I use a system prompt that explicitly discourages sycophancy, but this is a single sentence expression of preference and not an indication of fundamental model weakness.
* I’ll leave the anthropomorphism discussions to Searle; empirically this is the observed output.
Re: Ask HN: How do you deal with people who trust LLMs?
#43A friend of mine severely injured her leg, especially knee, and went through a surgery. She said that she had a rehabilitation plan for the next 6 months. Guess what, from Gemini. I just told her just listen to her doctor.
I didn't tell her why LLMs can make mistakes or hallucinate because I thought that she would not appreciate my mansplaining.
Looking forward though, my boring answer would still be education. It is going to take time. But without understanding LLMs, they will not be easily persuaded.
Re: Ask HN: How do you deal with people who trust LLMs?
#44Is this any different than people who believe random things they read on sketchy news sites or social media?
Yes, I think AI bots are more compelling to some people. They break the concept of judging information by its source because they obscure the source. But at the same time they are trained on a lot of reputable sources and can say a lot of very smart things, just at other times they say complete BS. But they are really good at making things sound plausible, that's essentially how they work after all.
I would argue for many people social media and news aggregators do the exact same thing. People site Instagram or TikTok as the source of their data in the same way others site ChatGPT
Re: Ask HN: How do you deal with people who trust LLMs?
#45tell them what to prompt the AI with to get the correct results. I've seen a number youtube shorts lately doing this, where some scientist gets "refuted" by some random person based on an LLM result, they then sit with the LLM and ask the same question, get the same wrong answer, then follow it up with a clarifying question, which then the LLM realizes its mistake and gives a better answer.
And then ask another question, and the LLM changes its mind again ("are you sure?").
It's not actually realizing anything so much as it's following your lead. Yes, followup questions can help dislodge more information, but fundamentally you can accidentally or on purpose bully an LLM to contradict itself quite easily, and it is only incidentally about correctness.
Re: Ask HN: How do you deal with people who trust LLMs?
#46Yesterday, I was praying to ChatGPT and asking for guidance on my car washing problem. Through its holy scripture, it suggested me to walk to the car wash to improve my fitness. When I arrived and found the absence of my car to be a true hindrance for washing, it occurred to me that I should have pondered the scripture more carefully to identify its true meaning. I treat the LLM like a diety. Every sane person unders…
I heard this exact story recently in a YouTube video, well, the car wash part at least. That was you?
It's an example people use to show the (current) limits of LLM capabilities. Like counting the 'r's in strawberry around two years ago.
Re: Ask HN: How do you deal with people who trust LLMs?
#47[dead]
Re: Ask HN: How do you deal with people who trust LLMs?
#48Simply prove them wrong (earnestly and in good faith). When they realise the LLM is fallible, they'll learn to be skeptical of it without you needing to teach them that specific lesson.
Re: Ask HN: How do you deal with people who trust LLMs?
#49Not that I've had to deal with this specifically, but I have noticed how the input phrasing in my prompts pushes the LLM in different directions. I've just tried a quick test with `duck.ai` on gpt 4o-mini with:
A: Why is drinking coffee every day so good for you?
B: Why is drinking coffee every day so bad for you?
Question A responds that it has "several health benefits", antioxidants, liver health, reduced risk of diabetes and Parkinson's.
Question B responds that it may lead to sleep disruption, digestive issues, risk of osteoporosis.
Same question. One word difference. Two different directions.
This makes me take everything with a pinch of salt when I ask "Would Library A be a good fit for Problem X" - which is obviously a bit leading; I don't even trust what I hope are more neutral inputs like "How does Library A apply to Problem Space X", for example.
Re: Ask HN: How do you deal with people who trust LLMs?
#50Yesterday, I was praying to ChatGPT and asking for guidance on my car washing problem. Through its holy scripture, it suggested me to walk to the car wash to improve my fitness. When I arrived and found the absence of my car to be a true hindrance for washing, it occurred to me that I should have pondered the scripture more carefully to identify its true meaning. I treat the LLM like a diety. Every sane person unders…
I heard this exact story recently in a YouTube video, well, the car wash part at least. That was you?