Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How I use Claude Code: Separation of planning and execution

boristane.com

Post reply on HN
Print view View original

181–190 of 630 posts

Re: How I use Claude Code: Separation of planning and execution

#181

Post by wazHFsRy »

wokwokwok wrote: post #151

This is the way. The practice is: - simple - effective - retains control and quality Certainly the “unsupervised agent” workflows are getting a lot of attention right now, but they require a specific set of circumstances to be effective: - clear validation loop (eg. Compile the kernel, here is gcc that does so correctly) - ai enabled tooling (mcp / cli tool that will lint, test and provide feedback immediately) - ove…

Absolutely. And you can also always let the agent look back at the plan to check if it is still on track and aligned.

One step I added, that works great for me, is letting it write (api-level) tests after planning and before implementation. Then I’ll do a deep review and annotation of these tests and tweak them until everything is just right.

Re: How I use Claude Code: Separation of planning and execution

#182

Post by [deleted] »

wokwokwok wrote: post #151

This is the way. The practice is: - simple - effective - retains control and quality Certainly the “unsupervised agent” workflows are getting a lot of attention right now, but they require a specific set of circumstances to be effective: - clear validation loop (eg. Compile the kernel, here is gcc that does so correctly) - ai enabled tooling (mcp / cli tool that will lint, test and provide feedback immediately) - ove…

[deleted]

Re: How I use Claude Code: Separation of planning and execution

#183

Post by basch »

wokwokwok wrote: post #151

This is the way. The practice is: - simple - effective - retains control and quality Certainly the “unsupervised agent” workflows are getting a lot of attention right now, but they require a specific set of circumstances to be effective: - clear validation loop (eg. Compile the kernel, here is gcc that does so correctly) - ai enabled tooling (mcp / cli tool that will lint, test and provide feedback immediately) - ove…

Honesty this is just language models in general at the moment, and not just coding.

It’s the same reason adding a thinking step works.

You want to write a paper, you have it form a thesis and structure first. (In this one you might be better off asking for 20 and seeing if any of them are any good.) You want to research something, first you add gathering and filtering steps before synthesis.

Adding smarter words or telling it to be deeper does work by slightly repositioning where your query ends up in space.

Asking for the final product first right off the bat leads to repetitive verbose word salad. It just starts to loop back in on itself. Which is why temperature was a thing in the first place, and leads me to believe they’ve turned the temp down a bit to try and be more accurate. Add some randomness and variability to your prompts to compensate.

Re: How I use Claude Code: Separation of planning and execution

#184

Post by manmal »

nostrademons wrote: post #85
haolez wrote: post #12

> Notice the language: “deeply”, “in great details”, “intricacies”, “go through everything”. This isn’t fluff. Without these words, Claude will skim. It’ll read a file, see what a function does at the signature level, and move on. You need to signal that surface-level reading is not acceptable. This makes no sense to my intuition of how an LLM works. It's not that I don't believe this works, but my mental model doesn…

It's the attention mechanism at work, along with a fair bit of Internet one-up-manship. The LLM has ingested all of the text on the Internet, as well as Github code repositories, pull requests, StackOverflow posts, code reviews, mailing lists, etc. In a number of those content sources, there will be people saying "Actually, if you go into the details of..." or "If you look at the intricacies of the problem" or "If yo…

I don’t think this is a result of the base training data („the internet“). It’s a post training behavior, created during reinforcement learning. Codex has a totally different behavior in that regard. Codex reads per default a lot of potentially relevant files before it goes and writes files.

Maybe you remember that, without reinforcement learning, the models of 2019 just completed the sentences you gave them. There were no tool calls like reading files. Tool calling behavior is company specific and highly tuned to their harnesses. How often they call a tool, is not part of the base training data.

Re: How I use Claude Code: Separation of planning and execution

#185

Post by basch »

trueno wrote: post #77
chickensong wrote: post #55

Earlier quoted context omitted.

For Claude at least, the more recent guidance from Anthropic is to not yell at it. Just clear, calm, and concise instructions.

wait seriously? lmfao thats hilarious. i definitely treat claude like shit and ive noticed the falloff in results. if there's a source for that i'd love to read about it.

If you think about where in the training data there is positivity vs negativity it really becomes equivalent to having a positive or negative mindset regarding a standing and outcome in life.

Re: How I use Claude Code: Separation of planning and execution

#186

Post by scuff3d »

tokioyoyo wrote: post #163
scuff3d wrote: post #146

Earlier quoted context omitted.

That's not how it works. It's not on everyone else to prove claims false, it's on you (or the people who argue any of this had a measurable impact) to prove it actually works. I've seen a bunch of articles like this, and more comments. Nobody I've ever seen has produced any kind of measurable metrics of quality based on one approach vs another. It's all just vibes. Without something quantifiable it's not much better…

Do you actively use LLMs to do semi-complex coding work? Because if not, it will sound mumbo-jumbo to you. Everyone else can nod along and read on, as they’ve experienced all of it first hand.

You've missed the point. This isn't engineering, it's gambling.

You could take the exact same documents, prompts, and whatever other bullshit, run it on the exact same agent backed by the exact same model, and get different results every single time. Just like you can roll dice the exact same way on the exact same table and you'll get two totally different results. People are doing their best to constrain that behavior by layering stuff on top, but the foundational tech is flawed (or at least ill suited for this use case).

That's not to say that AI isn't helpful. It certainly is. But when you are basically begging your tools to please do what you want with magic incantations, we've lost the fucking plot somewhere.

Re: How I use Claude Code: Separation of planning and execution

#187

Post by throwaway7783 »

I have to give this a try. My current model for backend is the same as how author does frontend iteration. My friend does the research-plan-edit-implement loop, and there is no real difference between the quality of what I do and what he does. But I do like this just for how it serves as documentation of the thought process across AI/human, and can be added to version control. Instead of humans reviewing PRs, perhaps humans can review the research/plan document.

On the PR review front, I give Claude the ticket number and the branch (or PR) and ask it to review for correctness, bugs and design consistency. The prompt is always roughly the same for every PR. It does a very good job there too.

Modelwise, Opus 4.6 is scary good!

Re: How I use Claude Code: Separation of planning and execution

#188

Post by computomatic »

haolez wrote: post #12

> Notice the language: “deeply”, “in great details”, “intricacies”, “go through everything”. This isn’t fluff. Without these words, Claude will skim. It’ll read a file, see what a function does at the signature level, and move on. You need to signal that surface-level reading is not acceptable. This makes no sense to my intuition of how an LLM works. It's not that I don't believe this works, but my mental model doesn…

If I say “you are our domain expert for X, plan this task out in great detail” to a human engineer when delegating a task, 9 times out of 10 they will do a more thorough job. It’s not that this is voodoo that unlocks some secret part of their brain. It simply establishes my expectations and they act accordingly.

To the extent that LLMs mimic human behaviour, it shouldn’t be a surprise that setting clear expectations works there too.

Re: How I use Claude Code: Separation of planning and execution

#189

Post by silversmith »

klipt wrote: post #140
jcdavis wrote: post #25

Earlier quoted context omitted.

Its a wild time to be in software development. Nobody(1) actually knows what causes LLMs to do certain things, we just pray the prompt moves the probabilities the right way enough such that it mostly does what we want. This used to be a field that prided itself on deterministic behavior and reproducibility. Now? We have AGENTS.md files that look like a parent talking to a child with all the bold all-caps, double emph…

Sufficiently advanced technology has become like magic: you have to prompt the electronic genie with the right words or it will twist your wishes.

Light some incense, and you too can be a dystopian space tech support, today! Praise Omnissiah!
Post reply on HN