Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs work best when the user defines their acceptance criteria first

blog.katanaquant.com

Post reply on HN
Print view View original

321–330 of 460 posts

Re: LLMs work best when the user defines their acceptance criteria first

#321

Post by nemo44x »

andai wrote: post #275
nemo44x wrote: post #268

Earlier quoted context omitted.

LLMs are literally goal machines. It’s all they do. So it’s important that you input specific goals for them to work towards. It’s also why logically you want to break the problem into many small problems with concrete goals.

Do you only mean instruct-tuned LLMs? Or the base (pretrained) model too?

The entire system and the agent loop allows for more complex goal resolution. The LLM models language (obviously) and language is goal oriented so it models goal oriented language. It’s an emergent feature of the system.

Re: LLMs work best when the user defines their acceptance criteria first

#322

Post by wood_spirit »

satvikpendem wrote: post #304
wood_spirit wrote: post #294

Earlier quoted context omitted.

The current way of doing AI cannot be trusted. that doesn’t mean the future won’t herald a way of using what a transformer is good at - interfacing with humans - to translate to and interact with something that can be a lot more sound and objective.

You're falling into the extrapolation fallacy, there is no reason to think that the future won't have the same issues as today in terms of hallucinations. And even if they were solved, how would that even work? The world is not sound and objective.

It’s a thought experiment. I am not saying I believe it will happen.

But right now there are lots of domains where current lauded success is in treating something objective - like code - as tokens for an llm.

We could instead explore using transformers to translate human languages to a symbology that can be reasoned about and applied eg to code.

It’s the talk of conferences. But whether it works better than we have today, or whether it aligns with the incentives or the big players, is another matter

Re: LLMs work best when the user defines their acceptance criteria first

#323

Post by patrakov »

roarcher wrote: post #303
treetalker wrote: post #244

This is my experience with how LLMs "draft" legal arguments: at first glance, it's plausible — but may be, and often is, invalid, unsound, and/or ill-advised. The catch is that many judges lack the time, energy, or willingness to not only read the documents in detail, but also roll up their sleeves and dig into the arguments and cited authorities. (Some lack the skills, but those are extreme cases.) So the plausible…

> LLM use in litigation drafting is thus akin to insurgent/guerilla warfare: it take little time, energy, or thinking to create, yet orders of magnitude more to analyze and refute. The same goes for coding. I have coworkers who use it to generate entire PRs. They can crank out two thousand lines of code that includes tests "proving" that it works, but may or may not actually be nonsense, in minutes. And then some poo…

Evil voice: "I don't mind not getting credits for the story points. The story was AI-generated anyway."

Re: LLMs work best when the user defines their acceptance criteria first

#324

Post by andai »

tovej wrote: post #299
satvikpendem wrote: post #288

Earlier quoted context omitted.

Why do people keep insisting that LLMs don't follow a chain of reasoning process? Using the latest LLMs you can see exactly what they "think" and see the resultant output. Plausible code does not mean random code as you seem to imply, it means...code that could work for this particular situation.

Because they don't. The chain-of-reasoning feature is really just a way to get the LLM to prompt more. The fact that it generates these "thinking" steps does not mean it is using them for reasoning. It's most useful effect is making it seem to a human that there is a reasoning process.

Is this position axiomatic or falsifiable? What would it take to change your mind?

Re: LLMs work best when the user defines their acceptance criteria first

#325

Post by andai »

seba_dos1 wrote: post #313
tovej wrote: post #299

Earlier quoted context omitted.

Because they don't. The chain-of-reasoning feature is really just a way to get the LLM to prompt more. The fact that it generates these "thinking" steps does not mean it is using them for reasoning. It's most useful effect is making it seem to a human that there is a reasoning process.

I love how generating strings like "let me check my notes" is effective at ending up with somewhat better end results - it pushes the weights towards outputting text that appears to be written by someone who did check their notes :D

I can't remember which lecture it was, but a guy said "they don't think, they only seem to think, and they won't replace a substantial portion of human labor, they will only seem to do so" ;)

Re: LLMs work best when the user defines their acceptance criteria first

#326

Post by andai »

satvikpendem wrote: post #308
tovej wrote: post #299

Earlier quoted context omitted.

Because they don't. The chain-of-reasoning feature is really just a way to get the LLM to prompt more. The fact that it generates these "thinking" steps does not mean it is using them for reasoning. It's most useful effect is making it seem to a human that there is a reasoning process.

How would you determine humans have reasoning then, in a way that LLMs do not?

Or — here's a fun one — subjective experience.

Re: LLMs work best when the user defines their acceptance criteria first

#327

Post by Jweb_Guru »

ttoinou wrote: post #155
Jweb_Guru wrote: post #121

Earlier quoted context omitted.

It's not reality. I'm really not a fan of the way that people excuse the really terrible code LLMs write by claiming that people write code just as bad. Even if that were true, it is not true that when you ask those people to do otherwise they simply pretend to have done it and forget you asked later.

No but they will despise you for bringing the problem up

In the long run, good code makes everyone much happier than code that is bad because people are being "nice" and letting things slide in code review to avoid confrontation.

Re: LLMs work best when the user defines their acceptance criteria first

#328

Post by seba_dos1 »

andai wrote: post #325
seba_dos1 wrote: post #313

Earlier quoted context omitted.

I love how generating strings like "let me check my notes" is effective at ending up with somewhat better end results - it pushes the weights towards outputting text that appears to be written by someone who did check their notes :D

I can't remember which lecture it was, but a guy said "they don't think, they only seem to think, and they won't replace a substantial portion of human labor, they will only seem to do so" ;)

Joking aside, this is exactly what happens with companies announcing "AI" replacing human labor when what they actually do is correcting for COVID-time overhiring while trying to make it appear in a way that won't make the stocks go too red.

Re: LLMs work best when the user defines their acceptance criteria first

#329

Post by jghn »

_0ffh wrote: post #320
jghn wrote: post #311

Earlier quoted context omitted.

> It used to be every "coder" was an "architect" as well, and did their own legwork as needed. I disagree. I remember in the days before "software engineer" became the rage that the standard job titles had a clear delineation between the people who thought the big thoughts with titles like "analyst" and the people who did the grunt work of coding who were "programmers". You'd also see roles in between like "programme…

Might be a big company thing then, but I'm not wholly convinced. There's a big gap between designing the outline of a big system and coding instructions that can be followed without having to make your own decisions. The question of how much of that gap is filled by the "design" vs "coding" levels is a spectrum.

I think I see what you're saying and if so we're talking past each other a bit and I agree with what you're saying as well.

The point I was raising is by the time an IC developer sees something, there's already been a process of curation that happens that frames the possible solutions & constrains branch points. This is different from saying that an IC makes 0 implementation decisions. The C-suite has set a direction. A product manager has defined the shape of the solution. A tech lead, architect, or whatever may have further limited scope. And any of these could just already be in effect at a global scale or on the specific problem at hand. Then the IC picks up the work and proceeds to make the last mile decisions. And it's turtles all the way up. At almost all levels on the career ladder, there are people above and/or upstream of you who are pre-curating your potential decision tree.

As an analogy, I once had a fresh tech lead under me where they didn't understand this. Their team became a mess. They'd introduce raw tickets straight from the PM to their team without having thought about them at all and things ground to a halt due to decision paralysis. From their perspective that's how it was always done when they were an IC in that group. The team tackled the tickets together to work out how to accomplish their goals. It took a lot of effort to convince them that what they *didn't see* was their prior tech lead narrowing down the search space a bit, and then framing the problem in a way that that made it easier for the team to move forward.

Re: LLMs work best when the user defines their acceptance criteria first

#330

Post by valicord »

jghn wrote: post #214
_0ffh wrote: post #202

Earlier quoted context omitted.

Huh, that explains a lot about the F500, and their buzzword slogans like "culture of excellence". LLM code is still mostly absurdly bad, unless you tell it in painstaking detail what to do and what to avoid, and never ask it to do a bigger job at a time than a single function or very small class. Edit: I'll admit though that the detailed explanation is often still much less work than typing everything yourself. But i…

> unless you tell it in painstaking detail what to do and what to avoid, and never ask it to do a bigger job at a time than a single function or very small class. This is hyperbolic, but the general sentiment is accurate enough, at least for now. I've noticed a bimodal distribution of quality when using these tools. The people who approach the LLM from the lens of a combo architect & PM, do all the leg work, set up t…

I agree with your first paragraph but not the second one. In many cases it's easier for me to directly write the code that satisfies the unwritten acceptance criteria I have in my head than to write those criteria down in English, have an LLM turn them into code, and then have to carefully review that code to see if I forgot some detail that changes everything.
Post reply on HN