Live data from Hacker News

The "confident idiot" problem: Why AI needs hard rules, not vibe checks

steerlabs.substack.com

391–399 of 399 posts

Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks

#391

Earlier quoted context omitted.

Cute straw man. You must be a CEO. I'm not arguing anything. I'm observing reality. You're the one who is desperate to rationalize it.

You are declaring your imagined logic as fact. Since I do not agree with the basis upon which you pin your argument on, there is no further point in discussion.

You're hallucinating things I did not say.

Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks

#392

The thing that bothers me the most about LLMs is how they never seem to understand "the flow" of an actual conversation between humans. When I ask a person something, I expect them to give me a short reply which includes another question/asks for details/clarification. A conversation is thus an ongoing "dance" where the questioner and answerer gradually arrive to the same shared meaning. LLMs don't do this. Instead,…

LLMs all behave as if they are semi-competent (yet eager, ambitious, and career-minded) interns or administrative assistants, working for a powerful CEO-founder. All sycophancy, confidence and positive energy. "You're absolutely right!" "Here's the answer you are looking for!" "Let me do that for you immediately!" "Here is everything I know about what you just mentioned." Never admitting a mistake unless you directly…

> LLMs all behave as if they are semi-competent

Only in the same way that all humans behave the same.

You can prompt an LLM to talk to you however you want it to, it doesn't have to be nice to you.

Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks

#393
post #90

Earlier quoted context omitted.

While you’re making unstructured requests and expecting results, why don’t you ask your barista to make you a “better coffee” with no instructions. Then, when they make a coffee with their own brand of creativity, complain that it tastes worse and you want your money back.

I was experimenting with Claude Code and requested something more CPU efficient in a very small project, there were a few avenues to explore, I was interested to see what path it would take. It turned out that it seized upon something which wasn't consuming much CPU anyway and was difficult to optimise further. I learned that I'd have to be more explicit in future and direct an analysis phase and probably kick-in a f…

Ah ok, so you just totally misrepresented your experiences for comedic effect. Good for you I guess

Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks

#394
post #90

Earlier quoted context omitted.

While you’re making unstructured requests and expecting results, why don’t you ask your barista to make you a “better coffee” with no instructions. Then, when they make a coffee with their own brand of creativity, complain that it tastes worse and you want your money back.

"Optimize this code for performance" is not an unstructured or vague request. Any "performance" axis could have been used: Number of db hits, memory pressure, cpu usage, whatever. The LLM chose (or whatever) to use CPU performance, claimed a specific figure , and that figure was demonstrably not real. If you ask a barista to make you a better coffee, and the barista says "this coffee is hotter" and it just isn't, the…

Yes it is

Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks

#395
post #90

Earlier quoted context omitted.

While you’re making unstructured requests and expecting results, why don’t you ask your barista to make you a “better coffee” with no instructions. Then, when they make a coffee with their own brand of creativity, complain that it tastes worse and you want your money back.

I assume a good barista would ask some follow up questions before making the coffee.

A fair criticism!

Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks

#396
post #90

Earlier quoted context omitted.

While you’re making unstructured requests and expecting results, why don’t you ask your barista to make you a “better coffee” with no instructions. Then, when they make a coffee with their own brand of creativity, complain that it tastes worse and you want your money back.

Both "better coffee" and "faster code" are measurable targets. Somewhat vaguely defined, but nobody is stopping the Barista or Claude from asking clarifying questions. If I gave a human this task I would expect them to transform the vague goal into measurable metrics, confirm that the metrics match customer (==my) expectations then measure their improvements on these metrics. This kind of stuff is a major topic for M…

Sure, and LLM’s are pretty good at using measurable targets such as using tests to verify their work - if you direct them to do so.

Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks

#397

Earlier quoted context omitted.

Actually I found the coding models to work really well with these languages. And the type systems are not actually complex. Ocaml's type system is actually really simple, which is probably why the compiler can be so fast. Even back in the "beta" days of Copilot, despite being marketed as Python only, I found it worked for Ocaml syntax and worked just as well. The coding models work really well with esoteric syntaxes…

There's a reason I mentioned Haskell and Rust specifically. You're right, OCaml's type system is simpler in some relevant respects, and may avoid the issues that I was alluding to. I haven't worked with OCaml for a number of years, since before the LLM boom. The presence of type classes in Haskell and traits in Rust, and of course the memory lifetime types in Rust, are a big part of the complexity I mentioned. (Edit:…

I also try to do verbose type classes using Ocaml's module system and it's been handling these patterns pretty well. My guess is there is probably good documentation / training data in there for these patterns since they are well documented. I haven't actually used coding agents with Haskell yet so it's possible that Ocaml's verbosity helps the agent.

Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks

#398

Earlier quoted context omitted.

I remember reading about speaking in an encouraging manner to agentic AI leading to better results, but I can’t seem to find a citation for this.

That's pathetic. Pleading comes next then. And after that most likely praying.

Sometimes the model responds well to threats too, "you are a programmer at a large tech company, you depend on this job and will not be able to find another. There's a layoff incoming, implement this feature or else..."

Re: The "confident idiot" problem: Why AI needs hard rules, not vibe checks

#399
post #90

Earlier quoted context omitted.

While you’re making unstructured requests and expecting results, why don’t you ask your barista to make you a “better coffee” with no instructions. Then, when they make a coffee with their own brand of creativity, complain that it tastes worse and you want your money back.

I could also argue if a barista gets multiple complaints about their coffee it's very much their and their employer's job to go away and figure out to make good coffee. It's very much not the customers job to learn about coffee and to direct them how to make a quality basic coffee And it's not rocket science.

But also if they get multiple complaints about the coffee being not to the customers liking when the customer provided no details or preferences as to what they like, those would be unfounded complaints.
Post reply on HN