Earlier quoted context omitted.
It’s easy to get AI to write bad code. Turns out you still need coding skills to get AI to write good code. But those who have figured it out can crank out working systems at a shocking pace.
I am sorry for asking, but... is there guide even on how to "figure it out"? Otherwise, how are you so sure about it?
LLMs work best when the user defines their acceptance criteria first
221–230 of 460 posts
Re: LLMs work best when the user defines their acceptance criteria first
#222Their default solution is to keep digging. It has a compounding effect of generating more and more code. If they implement something with a not-so-great approach, they'll keep adding workarounds or redundant code every time they run into limitations later. If you tell them the code is slow, they'll try to add optimized fast paths (more code), specialized routines (more code), custom data structures (even more code).…
This is why I'm confused when people say it isn't ready to replace most of the programmer workforce.
You need to be technical, have good communication skills, have big picture vision, be organized, etc. If you are a staff level engineer, you basically feel like you don’t need anyone else.
OTOH i have been seeing even fairly technical engineering managers struggle because they can’t get the LLMs to execute because they don’t know how to ask it what to do.
Re: LLMs work best when the user defines their acceptance criteria first
#223Earlier quoted context omitted.
And it’s slower to review because you didn’t do the hard part of understanding the code as it was being written.
You're holding it wrong. Set the boundaries and guidelines before it starts working. Don't leave it space to do things you don't understand. ie: enforce conventions, set specific and measurable/verifiable goals, define skeletons of the resulting solutions if you want/can. To give an example. I do a lot of image similarity stuff and I wanted to test the Redis VectorSet stuff when it was still in beta and the PHP exten…
Re: LLMs work best when the user defines their acceptance criteria first
#224This article is great. And the blog-article headline is interesting, but wrong. LLM's don't in general write plausible code (as a rule) either. They just write code that is (semantically) similar to code (clusters) seen in its training data, and which haven't been fenced off by RLHF / RLVR. This isn't that hard to remember, and is a correct enough simplification of what generative LLMs actually do, without resorting…
Exactly. It’s also easy to find yourself in the out-of-distribution territory. Just ask for some tree-sitter queries and watch Gemini 3, Opus 4.5 and GLM 5 hallucinate new directives.
We are building everything right now with LLM agents as a primary user in mind and one of our principles is “hallucination driven development”. If LLMs hallucinate an interface to your product regularly, that is a desire path and you should create that interface.
Re: LLMs work best when the user defines their acceptance criteria first
#225bad input > bad output idk what to say, just because it's rust doesn't mean it's performant, or that you asked for it to be performant. yes, llms can produce bad code, they can also produce good code, just like people
yes, llms can produce bad code, they can also produce good code, just like people Over time, you develop a feel for which human coders tend to be consistently "good" or "bad". And you can eliminate the "bad". With an LLM, output quality is like a box of chocolates, you never know what you're going to get. It varies based on what you ask and what is in it's training data --- which you have no way to examine in advance…
you can make horrible images with photoshop that doesn't make photoshop bad
Re: LLMs work best when the user defines their acceptance criteria first
#226Earlier quoted context omitted.
> If you ask to unify the duplication, it'll say "No problem, here's a brand new metamock abstract adapter framework that has a superset of all feature sets, plus two new metamock drivers for the older and the newer code! Let me know if you want me to write tests for the new adapters." Nevermind the fact that it only migrated 3 out of 5 duplicated sections, and hasn’t deleted any now-dead code.
Sounds like my coworkers.
Re: LLMs work best when the user defines their acceptance criteria first
#227Traditional API contracts assume a human reads docs and writes code once. But when agents are calling agents, the "contract" needs to be machine-verifiable in real-time.
The pattern I've seen work: explicit acceptance criteria in API responses themselves. Not just status codes, but structured metadata: "This response meets JSON Schema v2.1, latency was 180ms, data freshness is 3 seconds."
Lets the calling agent programmatically verify "did I get what I paid for?" without human intervention. The measurement problem becomes the automation problem.
Similar to how distributed systems moved from "hope it works" to explicit SLOs and circuit breakers. Agents need that, but at the individual request level.
Re: LLMs work best when the user defines their acceptance criteria first
#228> SQLite is not primarily fast because it is written in C. Well.. that too, but it is fast because 26 years of profiling have identified which tradeoffs matter. Someone (with deep pockets to bear the token costs) should let Claude run for 26 months to have it optimize its Rust code base iteratively towards equal benchmarks. Would be an interesting experiment. The article points out the general issue when discussing L…
I suspect performance is not the only problem with the codebase though.
Re: LLMs work best when the user defines their acceptance criteria first
#229Re: LLMs work best when the user defines their acceptance criteria first
#230This article is great. And the blog-article headline is interesting, but wrong. LLM's don't in general write plausible code (as a rule) either. They just write code that is (semantically) similar to code (clusters) seen in its training data, and which haven't been fenced off by RLHF / RLVR. This isn't that hard to remember, and is a correct enough simplification of what generative LLMs actually do, without resorting…
"Plausible" sounds like the right word to me. (It would be a mistake to digress into these features of LLMs in an article where it isn't needed.)