Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs work best when the user defines their acceptance criteria first

blog.katanaquant.com

Post reply on HN
Print view View original

271–280 of 460 posts

Re: LLMs work best when the user defines their acceptance criteria first

#271

Post by tsunamifury »

strken wrote: post #249
seanmcdirmid wrote: post #247

Ok, I’ll bite: how is that different from humans?

Human behaviour is goal-directed because humans have executive function. When you turn off executive function by going to sleep, your brain will spit out dreams. Dream logic is famous for being plausible but unhinged. I have the feeling that LLMs are effectively running on dream logic, and everything we've done to make them reason properly is insufficient to bring them up to human level.

It’s amazing how much you get wrong here. As LLM attention layers are stacked goal functions.

What they lack is multi turn long walk goal functions — which is being solved to some degree by agents.

Re: LLMs work best when the user defines their acceptance criteria first

#273

Post by ZeroGravitas »

Does it work if you get the agent to throw away all of its actual implementation and start again from scratch, keeping all the learning and tests and feedback?

Gemini seems to try to get a lot of information upfront with questions and plans but people are famously bad at knowing what they want.

Maybe it should build a series of prototypes and spikes to check? If making code is cheap then why not?

Re: LLMs work best when the user defines their acceptance criteria first

#274

Post by [deleted] »

treetalker wrote: post #244

This is my experience with how LLMs "draft" legal arguments: at first glance, it's plausible — but may be, and often is, invalid, unsound, and/or ill-advised. The catch is that many judges lack the time, energy, or willingness to not only read the documents in detail, but also roll up their sleeves and dig into the arguments and cited authorities. (Some lack the skills, but those are extreme cases.) So the plausible…

[deleted]

Re: LLMs work best when the user defines their acceptance criteria first

#275

Post by andai »

nemo44x wrote: post #268
strken wrote: post #249

Earlier quoted context omitted.

Human behaviour is goal-directed because humans have executive function. When you turn off executive function by going to sleep, your brain will spit out dreams. Dream logic is famous for being plausible but unhinged. I have the feeling that LLMs are effectively running on dream logic, and everything we've done to make them reason properly is insufficient to bring them up to human level.

LLMs are literally goal machines. It’s all they do. So it’s important that you input specific goals for them to work towards. It’s also why logically you want to break the problem into many small problems with concrete goals.

Do you only mean instruct-tuned LLMs? Or the base (pretrained) model too?

Re: LLMs work best when the user defines their acceptance criteria first

#276

Post by enraged_camel »

pornel wrote: post #36

Their default solution is to keep digging. It has a compounding effect of generating more and more code. If they implement something with a not-so-great approach, they'll keep adding workarounds or redundant code every time they run into limitations later. If you tell them the code is slow, they'll try to add optimized fast paths (more code), specialized routines (more code), custom data structures (even more code).…

I have no idea what I'm doing differently because I haven't experienced this since Opus 4.5. Even with Sonnet 4.5, providing explicit instructions along the lines of "reuse code where sensible, then run static analysis tools at the end and delete unused code it flags" worked really well.

I always watch Opus work, and it is pretty good with "add code, re-read the module, realize some pre-existing code (either it wrote, or was already there) is no longer needed and delete it", even without my explicit prompts.

Re: LLMs work best when the user defines their acceptance criteria first

#277

Post by andai »

wood_spirit wrote: post #270
satvikpendem wrote: post #257

Earlier quoted context omitted.

By now, a few years after ChatGPT released, I don't think anyone is thinking AI is objective and right, all users have seen at least one instance of hallucination and simply being wrong.

Sorry I can think of so many counter examples. I also detect a lot of “well it hallucinates about subject X (that the person knows well, so can spot the hallucination)” but continue to trust it on subjects Y and Z (which the person knows less well so can’t spot the hallucinations). YMMV.

> Briefly stated, the Gell-Mann Amnesia effect works as follows. You open the newspaper to an article on some subject you know well. In Murray's case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward-reversing cause and effect. I call these the "wet streets cause rain" stories. Paper's full of them. In any case, you read with exasperation or amusement the multiple errors in a story-and then turn the page to national or international affairs, and read with renewed interest as if the rest of the newspaper was somehow more accurate about far-off Palestine than it was about the story you just read. You turn the page, and forget what you know.

-Michael Crichton

Re: LLMs work best when the user defines their acceptance criteria first

#278

Post by andai »

satvikpendem wrote: post #261

Oftentimes, plausible code is good enough, hence why people keep using AI to generate code. This is a distinction without a difference.

There appears to be a similar approach in UX... plausible user experience is close enough.

Re: LLMs work best when the user defines their acceptance criteria first

#279

Post by queenkjuul »

pornel wrote: post #217
Implicated wrote: post #88

Earlier quoted context omitted.

Not trying to be snarky, with all due respect... this is a skill issue. It's a tool. It's a wildly effective and capable tool. I don't know how or why I have such a wildly different experience than so many that describe their experiences in a similar manner... but... nearly every time I come to the same conclusion that the input determines the output. > If they implement something with a not-so-great approach, they'l…

My comment was a summary of the situation, not literal prompts I use. I absolutely realize the work needs to be adequately described and agents must be steered in the right direction. The results also vary greatly depending on the task and the model, so devs see different rates of success. On non-trivial tasks (like adding a new index type to a db engine, not oneshotting a landing page) I find that the time and effor…

Don't worry, Claude ignores my CLAUDE.md and eats crayons anyway

Re: LLMs work best when the user defines their acceptance criteria first

#280

Post by andai »

bitwize wrote: post #256

You: Claude, do you know how to program? Claude: No, but if you hum a few bars I can fake it! Except "faking it" turns out to be good enough, especially if you can fake it at speed and get feedback as to whether it works. You can then just hillclimb your way to an acceptable solution.

Iterative Faking™ — now with plausible-looking test suite!
Post reply on HN