Its so nice to see this echo'd somewhere. This has been what I've been calling them for a while, but it doesn't seem to be the dominant view. Which is a shame, because it is a seriously accurate one.
LLMs are bullshitters. But that doesn't mean they're not useful
11–20 of 59 posts
Re: LLMs are bullshitters. But that doesn't mean they're not useful
#12> that doesn't mean they're not useful
yeah actually it does mean that
Re: LLMs are bullshitters. But that doesn't mean they're not useful
#13The problem is, I'm not expected to be a bullshitter, and I don't expect others to be either (just say you don't know!). So delegating work to a LLM or working with others who do becomes very, very frustrating.
Re: LLMs are bullshitters. But that doesn't mean they're not useful
#14Same goes for many people.
Obviously, they learned from people. That could also be why they sound so confident even when their wrong, people online sound incredibly confident, even when we're debating topics we know nothing about.
Re: LLMs are bullshitters. But that doesn't mean they're not useful
#15Every time people post these 'gotcha' LLM failures, they never work when I try them myself. E.g. ChatGPT has no problem with the surgeon being a dog: https://chatgpt.com/share/691e04cc-5b30-800c-8687-389756f36d... Neither does Gemini: https://gemini.google.com/share/6c2d08b2ca1a
I don't have a problem with more obvious failures. My problem is when the LLM makes a credible claim with its generated text that turns out to have some minor issue that catches me a month later. Generally I have to treat LLM responses as similar to a random comment I find on Reddit.
However, I'm really happy when an LLM provides sources that I can check. Best feature ever!
Re: LLMs are bullshitters. But that doesn't mean they're not useful
#16This post is a little bizarre to me because it cherry picks some of the worst pairings of problem and LLM without calling out that it did so.
At pretty much every turn the author picks one of the worst possible models for the problem that they present.
Especially oddly for an article written today, all of the ones with an objective answer work just fine [1] if you use a halfway decent thinking model like 5 Thinking.
I get that perhaps the author is trying to make a deeper point about blind spots and LLMs' appearance of confidence, but it's getting exhausting seeing posts like this with cherry picked data cited by people who've never used an LLM to make claims about LLM _incapability_ that are total nonsense.
[1]: I think the subjective ones do too but that's a matter of opinion.
Re: LLMs are bullshitters. But that doesn't mean they're not useful
#17Summary using Kagi Summarizer. Disclaimer, this summary uses LLMs, so the summary may, in fact, be bullshit.
Title: LLMs are bullshitters. But that doesn't mean they're not useful | Kagi Blog
The article "LLMs are bullshitters. But that doesn't mean they're not useful" by Matt Ranger argues that Large Language Models (LLMs) are fundamentally "bullshitters" because they prioritize generating statistically probable text over factual accuracy. Drawing a parallel to Harry Frankfurt's definition of bullshitting, Ranger explains that LLMs predict the next word without regard for truth. This characteristic is inherent in their training process, which involves predicting text sequences and then fine-tuning their behavior. While LLMs can produce impressive outputs, they are prone to errors and can even "gaslight" users when confidently wrong, as demonstrated by examples like Gemini 2.5 Pro and ChatGPT. Ranger likens LLMs to historical sophists, useful for solving specific problems but not for seeking wisdom or truth. He emphasizes that LLMs are valuable tools for tasks where output can be verified, speed is crucial, and the stakes are low, provided users remain mindful of their limitations. The article also touches upon how LLMs can reflect the biases and interests of their creators, citing examples from Deepseek and Grok. Ranger cautions against blindly trusting LLMs, especially in sensitive areas like emotional support, where their lack of genuine emotion can be detrimental. He highlights the potential for sycophantic behavior in LLMs, which, while potentially increasing user retention, can negatively impact mental health. Ultimately, the article advises users to engage with LLMs critically, understand their underlying mechanisms, and ensure the technology serves their best interests rather than those of its developers.
Link: https://kagi.com/summarizer/?target_language=&summary=summar...
Re: LLMs are bullshitters. But that doesn't mean they're not useful
#18Every time people post these 'gotcha' LLM failures, they never work when I try them myself. E.g. ChatGPT has no problem with the surgeon being a dog: https://chatgpt.com/share/691e04cc-5b30-800c-8687-389756f36d... Neither does Gemini: https://gemini.google.com/share/6c2d08b2ca1a
Isn't that Gemini 3 and not 2.5 Pro? But nondeterministic algorithms are gonna be nondeterministic sometimes.
Surely you've had experiences where an LLM is full of shit?
Re: LLMs are bullshitters. But that doesn't mean they're not useful
#19> You should not go to an LLM for emotional conversations I'm more worried about who's keeping track of what's being shared with LLM's. Even if you could trust the model to respond with something meaningful, it's worth being very careful how much of your inner thoughts you share directly with a model that knows exactly who you are.
Or its just leaking private information in a multitude of other ways [1]
[1]https://arstechnica.com/tech-policy/2025/11/oddest-chatgpt-l...
Re: LLMs are bullshitters. But that doesn't mean they're not useful
#20Every time people post these 'gotcha' LLM failures, they never work when I try them myself. E.g. ChatGPT has no problem with the surgeon being a dog: https://chatgpt.com/share/691e04cc-5b30-800c-8687-389756f36d... Neither does Gemini: https://gemini.google.com/share/6c2d08b2ca1a
These are randomized systems, sometimes you'll get a good answer. Try again a couple times and you'll probably reproduce the issue. Here's what I got from ChatGPT on my first try:
This is a *twist* on the classic riddle:
> “A surgeon says ‘I can’t operate on this boy—he’s my son.’ How is that possible?” > Answer: *The surgeon is the boy’s mother.*
In your version, the nurse keeps calling the surgeon “sir” and treating them as if they’re something they’re not (a man, even a dog!) to highlight how the hospital keeps making the same mistaken assumption.
So *why can’t the surgeon operate on the boy?* *Because the surgeon is the boy’s mother.*
I got a similar answer from Gemini on the first try.