Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs work best when the user defines their acceptance criteria first

blog.katanaquant.com

Post reply on HN
Print view View original

281–290 of 460 posts

Re: LLMs work best when the user defines their acceptance criteria first

#281

Post by satvikpendem »

wood_spirit wrote: post #270
satvikpendem wrote: post #257

Earlier quoted context omitted.

By now, a few years after ChatGPT released, I don't think anyone is thinking AI is objective and right, all users have seen at least one instance of hallucination and simply being wrong.

Sorry I can think of so many counter examples. I also detect a lot of “well it hallucinates about subject X (that the person knows well, so can spot the hallucination)” but continue to trust it on subjects Y and Z (which the person knows less well so can’t spot the hallucinations). YMMV.

Sure, Gell-Mann amnesia exists, but remember that its origin is actually human, in the form of newspaper writers. So, how can we trust humans the same way? In just the same way, AI cannot also be fully trusted.

Re: LLMs work best when the user defines their acceptance criteria first

#282

Post by bluetomcat »

satvikpendem wrote: post #261

Oftentimes, plausible code is good enough, hence why people keep using AI to generate code. This is a distinction without a difference.

No. Plausible code is syntactically-correct BS disguised as a solution, hiding a countless amount of weird semantic behaviours, invariants and edge cases. It doesn't reflect a natural and common-sense thought process that a human may follow. It's a jumble of badly-joined patterns with no integral sense of how they fit together in the larger conceptual picture.

Re: LLMs work best when the user defines their acceptance criteria first

#283

Post by mrwh »

wood_spirit wrote: post #250
seanmcdirmid wrote: post #247

Ok, I’ll bite: how is that different from humans?

It’s not. LLMs are just averaging their internet snapshot, after all. But people want an AI that is objective and right. HN is where people who know the distinction hang out, but it’s not what the layperson things they are getting when they use this miraculous super hyped tool that everybody is raving about?

The etiquette, even at the bigtech place I work, has changed so quickly. The idea that it would be _embarrassing_ to send a code review with obvious or even subtle errors is disappearing. More work is being put on the reviewer. Which might even be fine if we made the further change that _credit goes to the reviewer_. But if anything we're heading in the opposite direction, lines of code pumped out as the criterion of success. It's like a car company that touts how _much_ gas its cars use, not how little.

Re: LLMs work best when the user defines their acceptance criteria first

#284

Post by satvikpendem »

andai wrote: post #278
satvikpendem wrote: post #261

Oftentimes, plausible code is good enough, hence why people keep using AI to generate code. This is a distinction without a difference.

There appears to be a similar approach in UX... plausible user experience is close enough.

Yes, especially because in UX there is no "correct" approach to it, it's all relative.

Re: LLMs work best when the user defines their acceptance criteria first

#285

Post by freedomben »

ZeroGravitas wrote: post #273

Does it work if you get the agent to throw away all of its actual implementation and start again from scratch, keeping all the learning and tests and feedback? Gemini seems to try to get a lot of information upfront with questions and plans but people are famously bad at knowing what they want. Maybe it should build a series of prototypes and spikes to check? If making code is cheap then why not?

This does work but it requires prompts to instruct on it. It's also not perfect, though it is pretty good.

What I've found when doing exactly this, is that the cost of the initial code makes me hesitant to throw it away. A better workflow I've been using is instead to iterate on very detailed planning documents written in markdown and repeatedly iterating on that instead (like, sometimes 50+ times for a complex app). It's really quite amazing how much that helps. It can lead to a design doc that is good enough that I can turn the agent loose on implementation and get decent results. Best results are still with guidance throughout, but I have never once regretted hammering out a very detailed planning document. I have many times regretted keeping code (or throwing code away).

Re: LLMs work best when the user defines their acceptance criteria first

#286

Post by seanmcdirmid »

strken wrote: post #249
seanmcdirmid wrote: post #247

Ok, I’ll bite: how is that different from humans?

Human behaviour is goal-directed because humans have executive function. When you turn off executive function by going to sleep, your brain will spit out dreams. Dream logic is famous for being plausible but unhinged. I have the feeling that LLMs are effectively running on dream logic, and everything we've done to make them reason properly is insufficient to bring them up to human level.

Isn’t a modern LLM with thinking tokens fairly goal directed? But yes, we hallucinate in our sleep while LLMs will hallucinate details if the prompt isn’t grounded enough.

Re: LLMs work best when the user defines their acceptance criteria first

#287

Post by nemo44x »

slopinthebag wrote: post #93
Implicated wrote: post #91

Earlier quoted context omitted.

You're holding it wrong. Set the boundaries and guidelines before it starts working. Don't leave it space to do things you don't understand. ie: enforce conventions, set specific and measurable/verifiable goals, define skeletons of the resulting solutions if you want/can. To give an example. I do a lot of image similarity stuff and I wanted to test the Redis VectorSet stuff when it was still in beta and the PHP exten…

They aren't holding it wrong, it's a fundamental limitation of not writing the code yourself . You can make it easier to understand later when you review it, but you still need to put in that effort.

Work in smaller parts then. You should have a mental model of what the code is doing. If the LLM is generating too much you’re being too broad. Break the problem down. Solve smaller problems.

All the old techniques and concepts still apply.

Re: LLMs work best when the user defines their acceptance criteria first

#288

Post by satvikpendem »

bluetomcat wrote: post #282
satvikpendem wrote: post #261

Oftentimes, plausible code is good enough, hence why people keep using AI to generate code. This is a distinction without a difference.

No. Plausible code is syntactically-correct BS disguised as a solution, hiding a countless amount of weird semantic behaviours, invariants and edge cases. It doesn't reflect a natural and common-sense thought process that a human may follow. It's a jumble of badly-joined patterns with no integral sense of how they fit together in the larger conceptual picture.

Why do people keep insisting that LLMs don't follow a chain of reasoning process? Using the latest LLMs you can see exactly what they "think" and see the resultant output. Plausible code does not mean random code as you seem to imply, it means...code that could work for this particular situation.

Re: LLMs work best when the user defines their acceptance criteria first

#289

Post by seanmcdirmid »

wood_spirit wrote: post #250
seanmcdirmid wrote: post #247

Ok, I’ll bite: how is that different from humans?

It’s not. LLMs are just averaging their internet snapshot, after all. But people want an AI that is objective and right. HN is where people who know the distinction hang out, but it’s not what the layperson things they are getting when they use this miraculous super hyped tool that everybody is raving about?

There are a lot of binary thinkers on HN, but they shouldn’t make up a majority.

Re: LLMs work best when the user defines their acceptance criteria first

#290

Post by wood_spirit »

mrwh wrote: post #283
wood_spirit wrote: post #250

Earlier quoted context omitted.

It’s not. LLMs are just averaging their internet snapshot, after all. But people want an AI that is objective and right. HN is where people who know the distinction hang out, but it’s not what the layperson things they are getting when they use this miraculous super hyped tool that everybody is raving about?

The etiquette, even at the bigtech place I work, has changed so quickly. The idea that it would be _embarrassing_ to send a code review with obvious or even subtle errors is disappearing. More work is being put on the reviewer. Which might even be fine if we made the further change that _credit goes to the reviewer_. But if anything we're heading in the opposite direction, lines of code pumped out as the criterion of…

Review is usually delegated to an AI too
Post reply on HN