Live data from Hacker News

LLMs work best when the user defines their acceptance criteria first

blog.katanaquant.com

191–200 of 460 posts

Re: LLMs work best when the user defines their acceptance criteria first

#191
post #36

Their default solution is to keep digging. It has a compounding effect of generating more and more code. If they implement something with a not-so-great approach, they'll keep adding workarounds or redundant code every time they run into limitations later. If you tell them the code is slow, they'll try to add optimized fast paths (more code), specialized routines (more code), custom data structures (even more code).…

It’s in the name, isn’t it?

Generative AI.

Re: LLMs work best when the user defines their acceptance criteria first

#192
post #101

Earlier quoted context omitted.

Sounds like my coworkers.

That's the reality nobody really wants to say.

Maybe, but it lets them pump out much, much more code than they otherwise would have been able to. That's the "100x" in their AI productivity multipliers.

Re: LLMs work best when the user defines their acceptance criteria first

#194

The difference for me recently Write a lambda that takes an S3 PUT event and inserts the rows of a comma separated file into a Postgres database. Naive implementation: download the file from s3 and do a bulk insert - it would have taken 20 minutes and what Claude did at first. I had to tell it to use the AWS sql extension to Postgres that will load a file directly from S3 into a table. It took 20 seconds. I treat cod…

Same pattern in data engineering generally. LLMs default to the obvious row-by-row or download-then-insert approach and you have to steer them toward the efficient path (COPY, bulk loaders, server-side imports). Once you name the right primitive, they execute it correctly, permissions and all, as you found.

The deeper issue is that "efficient ingest" depends heavily on context that's implicit in your setup: file sizes, partitioning, schema evolution expectations, downstream consumers. A Lambda doing direct S3-to-Postgres import is fine for small/occasional files, but if you're dealing with high-volume event-driven ingestion you'll hit connection pool pressure fast on RDS. At that point the conversation shifts to something like a queue buffer or moving toward a proper staging layer (S3 → Redshift/Snowflake/Databricks with native COPY or autoloader). The LLM won't surface that tradeoff unless you explicitly bring it up. It optimizes for the stated task, not for the unstated architectural constraints.

Re: LLMs work best when the user defines their acceptance criteria first

#195

I tried to make Claude Code, Sonnet 4.6, write a program that draws a fleur-de-lis. No exaggeration it floundered for an hour before it started to look right. It's really not good at tasks it has not seen before.

[flagged]

Re: LLMs work best when the user defines their acceptance criteria first

#196
post #185

Earlier quoted context omitted.

It's not reality. I'm really not a fan of the way that people excuse the really terrible code LLMs write by claiming that people write code just as bad. Even if that were true, it is not true that when you ask those people to do otherwise they simply pretend to have done it and forget you asked later.

"Even if that were true, it is not true that when you ask those people to do otherwise they simply pretend to have done it and forget you asked later." I admire your experience with people.

The point is, that's not the typical experience and people like that can be replaced. We don't willingly bring people like that on our teams, and we certainly don't aim to replace entire teams with clones of this terrible coworker prototype.

Re: LLMs work best when the user defines their acceptance criteria first

#197

Earlier quoted context omitted.

And it’s slower to review because you didn’t do the hard part of understanding the code as it was being written.

You're holding it wrong. Set the boundaries and guidelines before it starts working. Don't leave it space to do things you don't understand. ie: enforce conventions, set specific and measurable/verifiable goals, define skeletons of the resulting solutions if you want/can. To give an example. I do a lot of image similarity stuff and I wanted to test the Redis VectorSet stuff when it was still in beta and the PHP exten…

You are correct but developers are not yet ready to face it. The argument you'll always get is the flawed premise that it's less effort to write it yourself (While the same people work in teams that have others writing code for them every day of the week).

Re: LLMs work best when the user defines their acceptance criteria first

#198

Earlier quoted context omitted.

At least my comment hasn't been reviewed or written by a LLM. And in my French brain, code or codebase is countable and not uncountable.

As far as I've ever heard, "le code" used in a codebase is uncountable, like "le café" you'd put in a cup, so we would still say "meilleur que tout le code que j'ai vu en 20 ans" and not "meilleur que tous les codes que j'ai vus en 20 ans". There is a countable "code" (just like "un café" is either a place, or a cup of coffee, or a type of coffee), and "un code" would be the one used as a password or secret, as in "j…

> As far as I've ever heard, "le code" used in a codebase is uncountable

Now I can't get the Pulp Fuction dialog out of my head.

- Do you know what they call code in France?

- No

- Le code

Re: LLMs work best when the user defines their acceptance criteria first

#199
post #22
post #3

Yes plausible text prediction is exactly what it is. However, I wonder if the author included benchmarking in their prompt. It's not exactly fair to keep hidden requirements.

Attributing these to "hidden requirements" is a slippery slope. My own experience using Claude Code and similar tools tells me that "hidden requirements" could include: * Make sure DESIGN.md is up to date * Write/update tests after changing source, and make sure they pass * Add integration test, not only unit tests that mock everything * Don't refactor code that is unrelated to the current task ... These are not even…

The training data is full of ‘any’ so you will keep getting ‘any’ because that is the code the models have seen.

An interesting example of the training data overriding the context.

Post reply on HN