A Software Development Methodology for Disciplined LLM Collaboration • Hacker News Board

Re: A Software Development Methodology for Disciplined LLM Collaboration

#31

Post by ianbicking » Sat, Sep 06, 2025, 7:11 PM UTC

grork wrote: post #18

perrygeo wrote: post #16

There's some irony; far from handling the details, LLMs are forcing programmers to adopt hyper-detailed, disciplined practices. They've finally cajoled software developers into writing documentation! Worth noting we've always had the capacity to implement these practices to improve HUMAN collaboration, but rarely bothered.

We’ve ultimately decided to treat the models with more respect, nurturing, and collaborative support than we ever did our follow human keyboard smashers. Writing all the documentation, detailed guidance, allowing them multiple attempts, to help the LLMs be successful. But Brenda, the early in career new grad? “please read this poorly written, 5 year-old, incomplete wiki, and don’t ask me questions” I’ve been thinking…

Respect (or lack thereof) goes both ways: both the writer and reader. I have frequently felt disrespected by producing documentation, planning/etc that isn't read. In the end I mostly rely on oral transmission of knowledge because then at least I can read the room and know if I'm providing some value to people, and ultimately we're both trapped in the room together and have to invest the same amount of time.

The LLM isn't always smart, but it's always attentive. It rewards that effort in a way that people frequently don't. (Arguably this is a company culture issue, but it's also a widespread issue.)

Re: A Software Development Methodology for Disciplined LLM Collaboration

#32

Post by rsecora » Sat, Sep 06, 2025, 7:24 PM UTC

Back in the day, when business computing emerged (COBOL, Mainframes...), it appear the distinction between systems analysts and programmers. Analyst understand business needs, programmers implemented those specs in code.

Years later, the industry evolved to integrate both roles, and new methodologies and new roles appear.

Now humans write specs, and AI agents write code. Role separation is a principle of labor division since Plato.

Re: A Software Development Methodology for Disciplined LLM Collaboration

#33

Post by wheelerwj » Sat, Sep 06, 2025, 8:35 PM UTC

kanak8278 wrote: post #27

Can anyone help me on how to integrate this with Claude-Code? I went through it, I already follow few things manually but when I think of integrating most of the parts (not all), I don't know where should I put it for the Coding-LLM to understand. I fear if I put everything in Claude.md it will be just too much context for the CC.

In Claude code you can definitely put most of this into claude.md, much of it in a global claude.md and some in the project claude.md.

which part are you specifically uncertain about?

Re: A Software Development Methodology for Disciplined LLM Collaboration

#34

Post by wheelerwj » Sat, Sep 06, 2025, 8:38 PM UTC

ianbicking wrote: post #30

perrygeo wrote: post #16

There's some irony; far from handling the details, LLMs are forcing programmers to adopt hyper-detailed, disciplined practices. They've finally cajoled software developers into writing documentation! Worth noting we've always had the capacity to implement these practices to improve HUMAN collaboration, but rarely bothered.

My experience writing in a professional setting is that people mostly don't read what I write, and the more effort I put into being thorough the less likely that it will be read.

That is an interesting observation. You're correct, the LLM inheritly reads and digests ever token you offer it.

Re: A Software Development Methodology for Disciplined LLM Collaboration

#35

Post by perrygeo » Sat, Sep 06, 2025, 10:51 PM UTC

ianbicking wrote: post #31

grork wrote: post #18

Earlier quoted context omitted.

We’ve ultimately decided to treat the models with more respect, nurturing, and collaborative support than we ever did our follow human keyboard smashers. Writing all the documentation, detailed guidance, allowing them multiple attempts, to help the LLMs be successful. But Brenda, the early in career new grad? “please read this poorly written, 5 year-old, incomplete wiki, and don’t ask me questions” I’ve been thinking…

Respect (or lack thereof) goes both ways: both the writer and reader. I have frequently felt disrespected by producing documentation, planning/etc that isn't read. In the end I mostly rely on oral transmission of knowledge because then at least I can read the room and know if I'm providing some value to people, and ultimately we're both trapped in the room together and have to invest the same amount of time. The LLM…

Great framing of the problem. I do think it's a culture issue with "Agile" practices in particular - By design, there is no time budgeted for reading, writing, reflection, or discussion. Sprint, sprint, sprint.

In organizations that value innovation, people will spend time reading and writing. It's a positive feedback loop, almost a litmus test of quality of the work culture.

Re: A Software Development Methodology for Disciplined LLM Collaboration

#36

Post by rapind » Sun, Sep 07, 2025, 9:11 AM UTC

CuriouslyC wrote: post #5

The most important thing is to have a strong plan cycle in front of you agent work, if you do that, agents are very reliable. You need to have a deep research cycle that basically collects a covering set of code that might need to be modified for a feature, feeds it into gemini/gpt5 to get a broad codebase level understanding, then has a debate cycle on how to address it, with the final artifact being a hyper detaile…

> The planning phase will produce plans that claude can iterate on for 3+ hours in some cases, if you tell it to complete the entire task in one shot, and the robust test validation / change set analysis will catch agents solving an easier problem because they got frustrated or not following directions.

Don't you run into context nightmares though? I was coming up with very detailed plans (using zen to vet with other models), but I found claude just doing the wrong thing a lot of the time, ignoring and / or forgetting very specific instructions and rules, especially across context compactions.

There's this one time that really sticks out in my mind because I had to constantly correct it; when to use ->> versus -> and handle null / type checks with PostgreSQL JSONB. Vibe coders would miss this sort of thing with testing unless they knew that JSONB null is not the same as SQL NULL (and other types too). When working with nested data, you probably won't have test coverage for it. This is just one of many examples too.

Re: A Software Development Methodology for Disciplined LLM Collaboration

#37

Post by rapind » Sun, Sep 07, 2025, 9:23 AM UTC

ianbicking wrote: post #30

perrygeo wrote: post #16

There's some irony; far from handling the details, LLMs are forcing programmers to adopt hyper-detailed, disciplined practices. They've finally cajoled software developers into writing documentation! Worth noting we've always had the capacity to implement these practices to improve HUMAN collaboration, but rarely bothered.

My experience writing in a professional setting is that people mostly don't read what I write, and the more effort I put into being thorough the less likely that it will be read.

Agreed, and I would argue the super quick turnaround time an interactive discussion during a planning phase make this much more enjoyable.

I also enjoy discussing solutions with people in real time too. But writing documentation in a vacuum without any feedback or even knowing if someone will read the spec?? Soul draining stuff.

In fact, the best of both worlds would be having a discussion with someone else (real person) while an AI agent listens, takes notes, and provides feedback / insights using different models. Vetting your ideas etc.

Re: A Software Development Methodology for Disciplined LLM Collaboration

#38

Post by MndlshnDscpl » Sun, Sep 07, 2025, 11:43 AM UTC

rapind wrote: post #36

CuriouslyC wrote: post #5

The most important thing is to have a strong plan cycle in front of you agent work, if you do that, agents are very reliable. You need to have a deep research cycle that basically collects a covering set of code that might need to be modified for a feature, feeds it into gemini/gpt5 to get a broad codebase level understanding, then has a debate cycle on how to address it, with the final artifact being a hyper detaile…

> The planning phase will produce plans that claude can iterate on for 3+ hours in some cases, if you tell it to complete the entire task in one shot, and the robust test validation / change set analysis will catch agents solving an easier problem because they got frustrated or not following directions. Don't you run into context nightmares though? I was coming up with very detailed plans (using zen to vet with other…

Agreed 100%. For those of us who have already spent ungodly hours creating hyper-detailed specifications for AI, the take that this is the solution to working with AI coding agents seems ridiculously naive. For context, I've also seen this behavior in Claude Code, and despite initially being extremely bullish on the technology, it's almost convinced me that it just isn't ready for prime time no matter what the hucksters might tell you. When you start seeing this you quickly realize that it doesn't really matter how many guardrails you put in place, or how detailed your specification is, if your coding agent randomly decides to ignore your rules or specifications(even in 'brand new context' scenarios). I've lost track of how many times I've asked Claude why did you do this, when it expressly says to do the opposite in the Claude.md file(including words like 'important' or 'critical'), or a specification document that you read right before implementing with a brand new context. Naturally, Claude's reply will be some variation of 'You're absolutely right to call me out on this. I should have done it the way it was spelled out in the specification.'

Re: A Software Development Methodology for Disciplined LLM Collaboration

#39

Post by lukaslalinsky » Sun, Sep 07, 2025, 2:55 PM UTC

mehdibl wrote: post #9

The most important you need always to do: 1. Plan, review the plan. 2. Review the code during changes before even it finish and fix ASAP you see drift. 3. Then again review 4. Add tests & use all quality tools don't rely 100% on LLM. 5. Don't trust LLM reviews for own produced code as it's very biased. This is basic steps that you do as you like. Avoid FULL AUTOMATED AGENT pipeline where you review the code only at t…

Even before you plan, you need to feed it enough relevant context, to make sure the plan is not based on hallucinations it assumes about the system. The best approach I found:

1. Make it read relevant pieces of code and explain it to me

2. Explain my problem and ask it to come up with a plan, iterate if needed

3. Allow it to execute the plan, watch it do it, interrupt and correct when needed

4. Have it do code review using a sub agent, focusing on correctness, avoiding leftover code, etc

5. Then I review it myself

Re: A Software Development Methodology for Disciplined LLM Collaboration

#40

Post by CuriouslyC » Sun, Sep 07, 2025, 3:00 PM UTC

rapind wrote: post #36

CuriouslyC wrote: post #5

The most important thing is to have a strong plan cycle in front of you agent work, if you do that, agents are very reliable. You need to have a deep research cycle that basically collects a covering set of code that might need to be modified for a feature, feeds it into gemini/gpt5 to get a broad codebase level understanding, then has a debate cycle on how to address it, with the final artifact being a hyper detaile…

> The planning phase will produce plans that claude can iterate on for 3+ hours in some cases, if you tell it to complete the entire task in one shot, and the robust test validation / change set analysis will catch agents solving an easier problem because they got frustrated or not following directions. Don't you run into context nightmares though? I was coming up with very detailed plans (using zen to vet with other…

The key is to have each step have very detailed instructions, and tell claude to dispatch the appropriate domain expert subagent for each step with the specific instructions for that step. That keeps root context hot, and each subagent only gets the instructions it needs and has a fresh context.