Earlier quoted context omitted.
> but it seems to me that the overwhelmingly majority The overwhelming majority of what?
Of users. It's an implicit subject from the first sentence.
But how do they know that, if it's of all users?
The "confident idiot" problem: Why AI needs hard rules, not vibe checks
191–200 of 399 posts
Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks
#192The thing that bothers me the most about LLMs is how they never seem to understand "the flow" of an actual conversation between humans. When I ask a person something, I expect them to give me a short reply which includes another question/asks for details/clarification. A conversation is thus an ongoing "dance" where the questioner and answerer gradually arrive to the same shared meaning. LLMs don't do this. Instead,…
Cursor Plan mode works like this. It restricts the LLMs access to your environment and will allow you to iteratively ask and clarify and it’ll piece together a plan that it allows you to review before it takes any action.
ChatGPT deep research does this but it’s weird and forced because it asks one series of questions and then goes off to the races, spending a half hour or more building a report. It’s frustrating if you don’t know what to expect and my wife got really mad the first time she wasted a deep research request asking it “can you answer multiple series of questions?” Or some other functionality clarifying question.
I’ve found Crusor’s plan mode extremely useful, similar to having a conversation with a junior or offshore team member who is eager to get to work but not TOO eager. These tools are extremely useful we just need to get the guard rails and user experience correct.
Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks
#193Earlier quoted context omitted.
lol humans are non-deterministic too
Yeah, but not when they are expected to perform in a job role. Too much nondeterminism in that case leads to firing and replacing the human with a more deterministic one.
>but not when they are expected to perform in a job role
I mean, this is why any critical systems involving humans have hard coded checklists and do not depend on people 'just winging it'. We really suck at determinism.
Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks
#194Ironic considering how many LLMs are competing to be trained on Reddit . . . which is the biggest repository of confidently incorrect people on the entire Internet. And I'm not even talking politics.
I've lost count of how much stuff I've seen there related to things I can credibly professionally or personally speak to that is absolute, unadulterated misinformation and bullshit. And this is now LLM training data.
Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks
#195We are trying to fix probability with more probability. That is a losing game. Thanks for pointing out the elephant in the room with LLMs. The basic design is non-deterministic. Trying to extract "facts" or "truth" or "accuracy" is an exercise in futility.
The factuality problem with LLMs isn't because they are non-deterministic or statistically based, but simply because they operate at the level of words, not facts. They are language models. You can't blame an LLM for getting the facts wrong, or hallucinating, when by design they don't even attempt to store facts in the first place. All they store are language statistics, boiling down to "with preceding context X, mos…
I think this is why I get much more utility out of LLMs with writing code. Code can fail if the syntax is wrong; small perturbations in the text (e.g. add a newline instead of a semicolon) can lead to significant increases in the cost function.
Of course, once an LLM is asked to create a bespoke software project for some complex system, this predictability goes away, the trajectory of the tokens succumbs to the intrinsic chaos of code over multi-block length scales, and the result feels more arbitrary and unsatisfying.
I also think this is why the biggest evangelists for LLMs are programmers, while creative writers and journalists are much more dismissive. With human language, the length scale over which tokens can be predicted is much shorter. Even the "laws" of grammar can be twisted or ignored entirely. A writer picks a metaphor because of their individual reading/life experience, not because its the most probable or popular metaphor. This is why LLM writing is so tedious, anodyne, sycophantic, and boring. It sounds like marketing copy because the attention model and RL-HF encourage it.
Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks
#196The thing that bothers me the most about LLMs is how they never seem to understand "the flow" of an actual conversation between humans. When I ask a person something, I expect them to give me a short reply which includes another question/asks for details/clarification. A conversation is thus an ongoing "dance" where the questioner and answerer gradually arrive to the same shared meaning. LLMs don't do this. Instead,…
Claude doesn't really have this problem.
Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks
#197Earlier quoted context omitted.
> but for what purpose? I recently introduced a non-technical person to Claude Code, and this non-human behavior was a big sticking point. They tried to talk to Claude similar as to a human, presenting it one piece of information at a time. With humans this is generally beneficial, and they will either nod for you to continue or ask clarifying questions. With Claude this does not work well, you have to infodump as mu…
also, this is what chat-style interfaces encourage. Anything where the "enter" key sends the message instead of creating a paragraph block is just hell. I'm prompting Gemini, and I write: I have the following code, can you help me analyze it? but Gemnini is already generating output, usually saying "I'm waiting for you to enter the code"
Like many chat-style interfaces, it's typically shift-enter to insert a newline.
Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks
#198The thing that bothers me the most about LLMs is how they never seem to understand "the flow" of an actual conversation between humans. When I ask a person something, I expect them to give me a short reply which includes another question/asks for details/clarification. A conversation is thus an ongoing "dance" where the questioner and answerer gradually arrive to the same shared meaning. LLMs don't do this. Instead,…
LLMs all behave as if they are semi-competent (yet eager, ambitious, and career-minded) interns or administrative assistants, working for a powerful CEO-founder. All sycophancy, confidence and positive energy. "You're absolutely right!" "Here's the answer you are looking for!" "Let me do that for you immediately!" "Here is everything I know about what you just mentioned." Never admitting a mistake unless you directly…
This is partly true, partly false, partly false in the opposite direction, with various new models. You really need to keep updating and have tons of interactions regularly in order to speak intelligently on this topic.
Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks
#199Ironic considering how many LLMs are competing to be trained on Reddit . . . which is the biggest repository of confidently incorrect people on the entire Internet. And I'm not even talking politics. I've lost count of how much stuff I've seen there related to things I can credibly professionally or personally speak to that is absolute, unadulterated misinformation and bullshit. And this is now LLM training data.
One thing I've had to explain to many confused friends who use reddit is that many of the people presenting themselves as domain experts in subreddits related to fields like law, accounting, plumbing, electrical, construction, etc. have absolutely no connection to or experience in whatever the field is.
Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks
#200- Claude, please optimise the project for performance. o Claude goes away for 15 minutes, doesn't profile anything, many code changes. o Announces project now performs much better, saving 70% CPU. - Claude, test the performance. o Performance is 1% _slower_ than previous. - Claude, can I have a refund for the $15 you just wasted? o [Claude waffles], "no".
I’ve always found the hard numbers on performance improvement hilarious. It’s just mimicking what people say on the internet when they get performance gains
> It’s just mimicking what people say on the internet when they get performance gains
probably read bunch of junior/mid level resumes saying they optimized 90% of company by 80%