Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How I use Claude Code: Separation of planning and execution

boristane.com

Post reply on HN
Print view View original

141–150 of 630 posts

Re: How I use Claude Code: Separation of planning and execution

#141

Post by tabs_or_spaces »

My workflow is a bit different.

* I ask the LLM for it's understanding of a topic or an existing feature in code. It's not really planning, it's more like understanding the model first

* Then based on its understanding, I can decide how great or small to scope something for the LLM

* An LLM showing good understand can deal with a big task fairly well.

* An LLM showing bad understanding still needs to be prompted to get it right

* What helps a lot is reference implementations. Either I have existing code that serves as the reference or I ask for a reference and I review.

A few folks do it at my work do it OPs way, but my arguments for not doing it this way

* Nobody is measuring the amount of slop within the plan. We only judge the implementation at the end

* it's still non deterministic - folks will have different experiences using OPs methods. If claude updates its model, it outdates OPs suggestions by either making it better or worse. We don't evaluate when things get better, we only focus on things not gone well.

* it's very token heavy - LLM providers insist that you use many tokens to get the task done. It's in their best interest to get you to do this. For me, LLMs should be powerful enough to understand context with minimal tokens because of the investment into model training.

Both ways gets the task done and it just comes down to my preference for now.

For me, I treat the LLM as model training + post processing + input tokens = output tokens. I don't think this is the best way to do non deterministic based software development. For me, we're still trying to shoehorn "old" deterministic programming into a non deterministic LLM.

Re: How I use Claude Code: Separation of planning and execution

#142

Post by girvo »

cowlby wrote: post #54
jamesmcq wrote: post #43

Earlier quoted context omitted.

The key part of my comment is "correctly". Does it write maintainable code? Does it write extensible code? Does it write secure code? Does it write performant code? My experience has been it failing most of these. The code might "work", but it's not good for anything more than trivial, well defined functions (that probably appeared in it's training data written by humans). LLMs have a fundamental lack of understandin…

I find “maintainable code” the hardest bias to let go of. 15+ years of coding and design patterns are hard to let go. But the aha moment for me was what’s maintainable by AI vs by me by hand are on different realms. So maintainable has to evolve from good human design patterns to good AI patterns. Specs are worth it IMO. Not because if I can spec, I could’ve coded anyway. But because I gain all the insight and capabi…

> But the aha moment for me was what’s maintainable by AI vs by me by hand are on different realms. So maintainable has to evolve from good human design patterns to good AI patterns.

How do you square that with the idea that all the code still has to be reviewed by humans? Yourself, and your coworkers

Re: How I use Claude Code: Separation of planning and execution

#143

Post by ValentineC »

raw_anon_1111 wrote: post #75
throwawaytea wrote: post #70

Earlier quoted context omitted.

What if it's just slower so that your daily work fits within the paid tier they want?

It isn’t slower. I use my personal ChatGPT subscriptions with Codex for almost everything at work and use my $800/month company Claude allowance only for the tricky stuff that Codex can’t figure out. It’s never application code. It’s usually some combination of app code + Docker + AWS issue with my underlying infrastructure - created with whatever IAC that I’m using for a client - Terraform/CloudFormation or the CDK.…

Curious: what are some cases where it'd make sense to not pay for the 20x plan (which is $200/month), and provide a whopping $800/month pay-per-token allowance instead?

Re: How I use Claude Code: Separation of planning and execution

#144

Post by stealthyllama »

jamesmcq wrote: post #9

This all looks fine for someone who can't code, but for anyone with even a moderate amount of experience as a developer all this planning and checking and prompting and orchestrating is far more work than just writing the code yourself. There's no winner for "least amount of code written regardless of productivity outcomes.", except for maybe Anthropic's bank account.

There is a miscommunication happening, this entire time we all had surprisingly different ideas about what quality of work is acceptable which seems to account for differences of opinion on this stuff.

Re: How I use Claude Code: Separation of planning and execution

#146

Post by scuff3d »

fragmede wrote: post #111
scuff3d wrote: post #106

Earlier quoted context omitted.

How anybody can read stuff like this and still take all this seriously is beyond me. This is becoming the engineering equivalent of astrology.

Feel free to run your own tests and see if the magic phrases do or do not influence the output. Have it make a Todo webapp with and without those phrases and see what happens!

That's not how it works. It's not on everyone else to prove claims false, it's on you (or the people who argue any of this had a measurable impact) to prove it actually works. I've seen a bunch of articles like this, and more comments. Nobody I've ever seen has produced any kind of measurable metrics of quality based on one approach vs another. It's all just vibes.

Without something quantifiable it's not much better then someone who always wears the same jersey when their favorite team plays, and swears they play better because of it.

Re: How I use Claude Code: Separation of planning and execution

#147

Post by Scrapemist »

stingraycharles wrote: post #49
haolez wrote: post #12

> Notice the language: “deeply”, “in great details”, “intricacies”, “go through everything”. This isn’t fluff. Without these words, Claude will skim. It’ll read a file, see what a function does at the signature level, and move on. You need to signal that surface-level reading is not acceptable. This makes no sense to my intuition of how an LLM works. It's not that I don't believe this works, but my mental model doesn…

It’s actually really common. If you look at Claude Code’s own system prompts written by Anthropic, they’re littered with “CRITICAL (RULE 0):” type of statements, and other similar prompting styles.

Where can I find those?

Re: How I use Claude Code: Separation of planning and execution

#148

Post by xscott »

nostrademons wrote: post #85
haolez wrote: post #12

> Notice the language: “deeply”, “in great details”, “intricacies”, “go through everything”. This isn’t fluff. Without these words, Claude will skim. It’ll read a file, see what a function does at the signature level, and move on. You need to signal that surface-level reading is not acceptable. This makes no sense to my intuition of how an LLM works. It's not that I don't believe this works, but my mental model doesn…

It's the attention mechanism at work, along with a fair bit of Internet one-up-manship. The LLM has ingested all of the text on the Internet, as well as Github code repositories, pull requests, StackOverflow posts, code reviews, mailing lists, etc. In a number of those content sources, there will be people saying "Actually, if you go into the details of..." or "If you look at the intricacies of the problem" or "If yo…

Of course I can't be certain, but I think the "mixture of experts" design plays into it too. Metaphorically, there's a mid-level manager who looks at your prompt and tries to decide which experts it should be sent to. If he thinks you won't notice, he saves money by sending it to the undergraduate intern.

Just a theory.

Re: How I use Claude Code: Separation of planning and execution

#149

Post by cowlby »

girvo wrote: post #142
cowlby wrote: post #54

Earlier quoted context omitted.

I find “maintainable code” the hardest bias to let go of. 15+ years of coding and design patterns are hard to let go. But the aha moment for me was what’s maintainable by AI vs by me by hand are on different realms. So maintainable has to evolve from good human design patterns to good AI patterns. Specs are worth it IMO. Not because if I can spec, I could’ve coded anyway. But because I gain all the insight and capabi…

> But the aha moment for me was what’s maintainable by AI vs by me by hand are on different realms. So maintainable has to evolve from good human design patterns to good AI patterns. How do you square that with the idea that all the code still has to be reviewed by humans? Yourself, and your coworkers

I picture like semi conductors; the 5nm process is so absurdly complex that operators can't just peek into the system easily. I imagine I'm just so used to hand crafting code that I can't imagine not being able to peek in.

So maybe it's that we won't be reviewing by hand anymore? I.e. it's LLMs all the way down. Trying to embrace that style of development lately as unnatural as it feels. We're obv not 100% there yet but Claude Opus is a significant step in that direction and they keep getting better and better.

Re: How I use Claude Code: Separation of planning and execution

#150

Post by FuckButtons »

haolez wrote: post #12

> Notice the language: “deeply”, “in great details”, “intricacies”, “go through everything”. This isn’t fluff. Without these words, Claude will skim. It’ll read a file, see what a function does at the signature level, and move on. You need to signal that surface-level reading is not acceptable. This makes no sense to my intuition of how an LLM works. It's not that I don't believe this works, but my mental model doesn…

That’s because it’s superstition.

Unless someone can come up with some kind of rigorous statistics on what the effect of this kind of priming is it seems no better than claiming that sacrificing your first born will please the sun god into giving us a bountiful harvest next year.

Sure, maybe this supposed deity really is this insecure and needs a jolly good pep talk every time he wakes up. or maybe you’re just suffering from magical thinking that your incantations had any effect on the random variable word machine.

The thing is, you could actually prove it, it’s an optimization problem, you have a model, you can generate the statistics, but no one as far as I can tell has been terribly forthcoming with that , either because those that have tried have decided to try to keep their magic spells secret, or because it doesn’t really work.

If it did work, well, the oldest trick in computer science is writing compilers, i suppose we will just have to write an English to pedantry compiler.

Post reply on HN