Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The quality of AI-assisted software depends on unit of work management

blog.nilenso.com

Post reply on HN
Print view View original

81–90 of 127 posts

Re: The quality of AI-assisted software depends on unit of work management

#81

Post by rglover »

What I've found works best:

1. Assume that any model will start to lose focus beyond 50K-100K tokens (even with a huge context window).

2. Be gluttonous with chats. At the first sign of confusion or mistakes, tell it to generate a new prompt and move to a new chat.

3. Write detailed prompts with clear expectations (from how the code should be written to the specific implementation that's required). Combine these with context like docs to get a fairly consistent hit rate.

4. Use tools like Cline that let you switch between an "Act" and "Plan" mode. This saves a ton of tokens but also avoids the LLM getting stuck on a loop when it's debugging.

I recently wrote this short blog post related to this: https://ryanglover.net/blog/treat-the-ai-like-it-s-yourself

The above approach helped me to implement a full-blown database wrapper around LMDB for Node.js in ~2 weeks of slow back-and-forth (link to code in post for those who are curious).

Re: The quality of AI-assisted software depends on unit of work management

#82

Post by chrisrickard »

We are tackling this at https://userdoc.fyi - we help you build your specs (epics, stories, acceptance criteria, tech notes, test cases, etc) - then you can generate what we call Dev Plans, one or more requirement layers for implementation.

e.g maybe a dev plan is all your authentication feature requirements, or in the house of analogy – all the requirements for the rooms, but with instructions to actually just first build the floor, and the walls.

Dev plans then slice the reqs into meaningful units of work, as mentioned in the article – a feature/story, is often too large of a checkpoint, or often needs to be implemented in collaboration with other features/stories, so it understands the correct architectural context,.

You can then implement Dev plans over MCP, or copy to .md for tools like Lovable or V0.

Re: The quality of AI-assisted software depends on unit of work management

#83

Post by ewoodrich »

bryanlarsen wrote: post #77
smw wrote: post #44

Earlier quoted context omitted.

Hell, claude even makes that part of the standard workflow, with /compact; cleverly using the llm itself to summarize the previous context

Once you see that message it's time to finish the task without AI because Claude will start crapping over your codebase if you let it continue.

I just got burned by that, it compacted and then immediately dropped what it was in the middle of working on to redo something it had already finished half an hour earlier. Which, predictably, sent it into "systematically destroy the entire working codebase" mode because the code it was now reading didn't match expectations of the original instructions. So it started extrapolating like "huh, it looks like the function already exists, therefore, user must have meant [bizarre completely out of left field guess] instead" in an escalating loop of confusion and code mangling.

Re: The quality of AI-assisted software depends on unit of work management

#84

Post by zmmmmm »

jmcodes wrote: post #79
angusturner wrote: post #15

Earlier quoted context omitted.

I think most SWEs do have a good idea where I work. They know that its a significant, but not revolutionary improvement. If you supervise and manage your agents closely on well scoped (small) tasks they are pretty handy. If you need a prototype and don't care about code quality or maintenance, they are great. Anyone claiming 2x, 5x, 10x etc is absolutely kidding themselves for any non-trivial software.

I've found a pretty good speed up just letting Claude Code run with a custom prompt to gather the context (relevant files, types, etc..) for the task then having it put together a document with that context. It takes all of five minutes to have it run and at the end I can review it, if it's small ask it to execute, and if it actually requires me to work it myself well now I have a reference with line numbers, some co…

> I've been a senior dev for a while now, a lot of my job _is_ reviewing other people's pull requests

I kind of hate that I'm saying this, but I'm sort of similar and one thing I really like is having zero guilt about trashing the LLM's code. So often people are submitting something and the code is OK but just pervasively not quite how I like it. Some staff will engage in micro arguments about things rather than just doing them how I want and it's just tiring. Then LLMs are really good at explaining why they did stuff (or simulating that) as well. LLMs will enthusiastically redo something and then help adjust their own AGENTS.md file to align better in the future.

Re: The quality of AI-assisted software depends on unit of work management

#85

Post by zmmmmm »

liszper wrote: post #5
jonstewart wrote: post #4

I first tried getting specific with Claude Code. I made the Claude.md, I detailed how to do TDD, what steps it should take, the commands it should run. It was imperfect. Then I had it plan (think hard) and write the plan to a file. I’d clear context, have it read the plan, ask me questions, and then have it decompose the plan into a detailed plan of discrete tasks. Have it work its way through that. It would inevitab…

With all due respect, you sound like someone who is just getting familiar with these tools. 100 more hours spent with AI coding and you will be much more productive. Coding with AI is a slightly different skill from coding, similar how managing software engineers is different from writing software.

ah, they are holding it wrong.

I am always so skeptical of this style of response. Because if it takes hundreds of hours to learn to use something, how can it really be the silver bullet everyone was claiming earlier? Surely they were all in the midst of the 100 hours. And what else could we do if we spent 100 hours learning something? It's a lot of time, a huge investment, all on faith that things will get better.

Re: The quality of AI-assisted software depends on unit of work management

#86

Post by zmmmmm »

I prefer small units of work. It really surprises me how fast people have leapt from the 2x speedup (mind blowing level of productivity increase) to full agentic coding without really questioning if it's a good idea.

When I have let Claude loose and vibe coded up hundreds of lines at a time that I have no familiarity with, I viscerally feel how I no longer understand or can maintain the app I've built. If I can't get Claude to do the next change I need, I'm screwed.

I'm very satisfied at the moment to be wielding LLMs as a tool at the individual function / microfeature level and getting a very satisfying productivity improvement.

Re: The quality of AI-assisted software depends on unit of work management

#87

Post by derefr »

dragonwriter wrote: post #53
nicce wrote: post #50

Earlier quoted context omitted.

> Expressing how I want something to be done in natural language is incredibly hard Surprise, surprise… that is why programming languages were created.

Programming languages don’t solve that problem, since someone still has to explain what needs to be done in natural language unless the end customer is also the programmer. Programming languages were created because of the different problem of “its very hard to get computers to understand natural language even if you know how to express what you want in it”.

> someone still has to explain what needs to be done in natural language unless the end customer is also the programmer

You're conflating requirement analysis with design. The customer only needs to describe the problem — a set of constraints on what comprises a valid solution. The software engineer is then free to design and develop a particular valid solution (and show it to the customer, which will result in more feedback, which will feed back into design, and so on.)

Formalizing this split is the premise behind Domain Driven Design (DDD): you can sit with the customer and pin down a problem description (= set of design requirements) together with them, expressed in exactly the natural language the customer-as-problem-domain-expert uses, without any reference to any particular potential design's solution-space domain. You can then turn around and reuse that set of natural-language statements as the skeleton of a test suite, that "enforces" the customer's expectations upon any potential design you create.

It's a lot like an artist sitting with a customer who's commissioning them, with the artist sketching something the customer is describing; and then the artist going away to actually illustrate/paint/craft/design/etc the thing, constrained by that sketch.

Re: The quality of AI-assisted software depends on unit of work management

#88

Post by cadamsdotcom »

jonstewart wrote: post #4

I first tried getting specific with Claude Code. I made the Claude.md, I detailed how to do TDD, what steps it should take, the commands it should run. It was imperfect. Then I had it plan (think hard) and write the plan to a file. I’d clear context, have it read the plan, ask me questions, and then have it decompose the plan into a detailed plan of discrete tasks. Have it work its way through that. It would inevitab…

Don’t give up on TDD.

I’ve invested hundreds of hours in process and tooling, and can now ship major features with tests in record time with Claude Code.

You have to coach it in TDD - no matter how much you explain in CLAUDE.md. That’s part because “a test that fails because the code isn’t written yet” is conceptually very similar to “a test that passes without the code we’re about to write” and is also similar to “a test that asserts the code we’re about to write is not there”. You have to watch closely to make sure it produces the first thing.

Why does it keep getting confused? You can’t blame it really. When two things are conceptually similar, models need lots of examples to distinguish between them. If the set of samples is sparse the model is likely to jump the small distance from a concept to similar ones.

So, you have to accept this as how Claude 4 works, keep it on a short leash, keep reminding it that it must watch the test fail, ask it if the test failed for the right reason (not some setup issue), and THEN give it permission to write the code.

The result is two mirror copies of your feature or fix: code and tests.

Reviewing code and tests together is pleasant because they mirror one another. The tests forever ensure your feature works as described, no manual testing needed, no regressions. And the model knows all the tricks to make your tests really beautiful.

TDD is the check and balance missing from most people’s agentic software dev process.

Re: The quality of AI-assisted software depends on unit of work management

#89

Post by hackable_sand »

igouy wrote: post #66
danparsonson wrote: post #58

I seem to be in a minority but I find user stories or features to be really awkward and unnatural units of work for building software. Sure these things help to define the expected result but they shouldn't directly drive the development process. Imagine building a house that way - you don't build the living room, then the kitchen, then the bathroom etc.; you build floors, walls, the roof... The 'features' or use cas…

Software is not like "building a house" and is not like a sculpture and is not like a cake because software is (mostly) notional not physical.

I don't see the difference. Could you explain how the physical attributes change the analogies?

Re: The quality of AI-assisted software depends on unit of work management

#90

Post by igouy »

hackable_sand wrote: post #89
igouy wrote: post #66

Earlier quoted context omitted.

Software is not like "building a house" and is not like a sculpture and is not like a cake because software is (mostly) notional not physical.

I don't see the difference. Could you explain how the physical attributes change the analogies?

The physical constraints govern the development processes described in the analogies.

The process for software is not constrained in that way.

Post reply on HN