Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs work best when the user defines their acceptance criteria first

blog.katanaquant.com

Post reply on HN
Print view View original

291–300 of 460 posts

Re: LLMs work best when the user defines their acceptance criteria first

#291

Post by Mooshux »

raw_anon_1111 wrote: post #231
datagobes wrote: post #194

Earlier quoted context omitted.

Same pattern in data engineering generally. LLMs default to the obvious row-by-row or download-then-insert approach and you have to steer them toward the efficient path (COPY, bulk loaders, server-side imports). Once you name the right primitive, they execute it correctly, permissions and all, as you found. The deeper issue is that "efficient ingest" depends heavily on context that's implicit in your setup: file size…

Also with Redshift - split the file up before ingestion to equal the number of nodes or combine a lot of small files into larger files before putting them into S3 and/or use an Athena CTAS command to combine a lot of small files into one big file. So in my other case, the whole thing was Web crawler (internal customer website) using Playwrite -> S3 -> SNS -> SQS -> Lambda (embed with Bedrock) -> S3 Vector Store. Simi…

[flagged]

Re: LLMs work best when the user defines their acceptance criteria first

#292

Post by satvikpendem »

rDr4g0n wrote: post #266
seanmcdirmid wrote: post #247

Ok, I’ll bite: how is that different from humans?

It's much easier to fire an employee which produces low quality/effort work than to convince leadership to fire Claude.

You can fire employees who don't review code generated though, because ultimately it's their responsibility to own their code, whether they hand wrote it or an LLM did.

It seems to me that it's all a matter of company culture, as it has always been, not AI. Those that tolerate bad code will continue to tolerate it, at their peril.

Re: LLMs work best when the user defines their acceptance criteria first

#293

Post by satvikpendem »

jghn wrote: post #214
_0ffh wrote: post #202

Earlier quoted context omitted.

Huh, that explains a lot about the F500, and their buzzword slogans like "culture of excellence". LLM code is still mostly absurdly bad, unless you tell it in painstaking detail what to do and what to avoid, and never ask it to do a bigger job at a time than a single function or very small class. Edit: I'll admit though that the detailed explanation is often still much less work than typing everything yourself. But i…

> unless you tell it in painstaking detail what to do and what to avoid, and never ask it to do a bigger job at a time than a single function or very small class. This is hyperbolic, but the general sentiment is accurate enough, at least for now. I've noticed a bimodal distribution of quality when using these tools. The people who approach the LLM from the lens of a combo architect & PM, do all the leg work, set up t…

It's almost as if architecture and code quality mattered just as before and that those who don't know proper engineering principles and problem decomposition will not succeed with these new tools.

Re: LLMs work best when the user defines their acceptance criteria first

#294

Post by wood_spirit »

satvikpendem wrote: post #281
wood_spirit wrote: post #270

Earlier quoted context omitted.

Sorry I can think of so many counter examples. I also detect a lot of “well it hallucinates about subject X (that the person knows well, so can spot the hallucination)” but continue to trust it on subjects Y and Z (which the person knows less well so can’t spot the hallucinations). YMMV.

Sure, Gell-Mann amnesia exists, but remember that its origin is actually human, in the form of newspaper writers. So, how can we trust humans the same way? In just the same way, AI cannot also be fully trusted.

The current way of doing AI cannot be trusted.

that doesn’t mean the future won’t herald a way of using what a transformer is good at - interfacing with humans - to translate to and interact with something that can be a lot more sound and objective.

Re: LLMs work best when the user defines their acceptance criteria first

#295

Post by tovej »

seanmcdirmid wrote: post #286
strken wrote: post #249

Earlier quoted context omitted.

Human behaviour is goal-directed because humans have executive function. When you turn off executive function by going to sleep, your brain will spit out dreams. Dream logic is famous for being plausible but unhinged. I have the feeling that LLMs are effectively running on dream logic, and everything we've done to make them reason properly is insufficient to bring them up to human level.

Isn’t a modern LLM with thinking tokens fairly goal directed? But yes, we hallucinate in our sleep while LLMs will hallucinate details if the prompt isn’t grounded enough.

Assuming this is not a rhetorical question: no, it is not. The only "goal" is to maximize plausibility.

Re: LLMs work best when the user defines their acceptance criteria first

#296

Post by zarzavat »

seanmcdirmid wrote: post #286
strken wrote: post #249

Earlier quoted context omitted.

Human behaviour is goal-directed because humans have executive function. When you turn off executive function by going to sleep, your brain will spit out dreams. Dream logic is famous for being plausible but unhinged. I have the feeling that LLMs are effectively running on dream logic, and everything we've done to make them reason properly is insufficient to bring them up to human level.

Isn’t a modern LLM with thinking tokens fairly goal directed? But yes, we hallucinate in our sleep while LLMs will hallucinate details if the prompt isn’t grounded enough.

The thing about dream logic is that it can be a completely rational series of steps, but there's usually a giant plot hole which you only realise the second you wake up.

This definitely matches my experience of talking to AI agents and chatbots. They can be extremely knowledgeable on arcane matters yet need to have obvious (to humans) assumptions pointed out to them, since they only have book smarts and not street smarts.

Re: LLMs work best when the user defines their acceptance criteria first

#297

Post by switchbak »

apical_dendrite wrote: post #252
seanmcdirmid wrote: post #247

Ok, I’ll bite: how is that different from humans?

The volume is different. Someone submitted a PR this week that was 3800 lines of shell script. Most of it was crap and none of it should have been in shell script. He's submitting PRs with thousands of lines of code every day. He has no idea how any of it actually works, and it completely overwhelms my ability to review. Sure, he could have submitted a ill-considered 3800 line PR five years ago, but it would have tak…

It’s harder when the person doing what you describe has the ability to have you fired. Power asymmetry + irresponsible AI use + no accountability = a recipe for a code base going right to hell in a few months.

I think we’re going to see a lot of the systems we depend on fail a lot more often. You’d often see an ATM or flight staus screen have a BSOD - I think we’re going to see that kind of thing everywhere soon.

Re: LLMs work best when the user defines their acceptance criteria first

#298

Post by tovej »

satvikpendem wrote: post #260
somewhereoutth wrote: post #253

Earlier quoted context omitted.

Humans have a 'world model' beyond the syntax - for code, an idea of what the code should do and how it does it. Of course, some humans are better than others at this, they are recognized as good programmers .

Papers show that AI also has a world model, so I don't think that's the right distinction.

Could you please cite these papers. If by AI you mean LLMs, that is not supported by what I know. If you mean a theoretical world-model-based AI, that's just a tautological statement.

Re: LLMs work best when the user defines their acceptance criteria first

#299

Post by tovej »

satvikpendem wrote: post #288
bluetomcat wrote: post #282

Earlier quoted context omitted.

No. Plausible code is syntactically-correct BS disguised as a solution, hiding a countless amount of weird semantic behaviours, invariants and edge cases. It doesn't reflect a natural and common-sense thought process that a human may follow. It's a jumble of badly-joined patterns with no integral sense of how they fit together in the larger conceptual picture.

Why do people keep insisting that LLMs don't follow a chain of reasoning process? Using the latest LLMs you can see exactly what they "think" and see the resultant output. Plausible code does not mean random code as you seem to imply, it means...code that could work for this particular situation.

Because they don't. The chain-of-reasoning feature is really just a way to get the LLM to prompt more.

The fact that it generates these "thinking" steps does not mean it is using them for reasoning. It's most useful effect is making it seem to a human that there is a reasoning process.

Re: LLMs work best when the user defines their acceptance criteria first

#300

Post by siliconc0w »

Just a recent anecdote, I asked the newest Codex to create a UI element that would persist its value on change. I'm using Datastar and have the manual saved on-disk and linked from the AGENTS.md. It's a simple html element with an annotation, a new backend route, and updating a data model. And there are even examples of this elsewhere in the page/app.

I've asked it to do why harder things so I thought it'd easily one-shot this but for some reason it absolutely ate it on this task. I tried to re-prompt it several times but it kept digging a hole for itself, adding more and more in-line javascript and backend code (and not even cleaning up the old code).

It's hard to appreciate how unintuitive the failure modes are. It can do things probably only a handful of specialists can do but it can also critical fail on what is a straightforward junior programming task.

Post reply on HN