Live data from Hacker News

Verified Spec-Driven Development (VSDD)

gist.github.com

101–110 of 122 posts

Re: Verified Spec-Driven Development (VSDD)

#101
post #54

Earlier quoted context omitted.

This. Waterfall never worked for a reason. Humans and agents both need to develop a first draft, then re-evaluate with the lessons learned and the structure that has evolved. It’s very very time consuming to plan a complex, working system up front. NASA has done it, for the moon landing. But we don’t have those resources, so we plan, build, evaluate, and repeat.

> Waterfall never worked for a reason We're going to need some evidence for this claim. I feel like nearly 70 years of NASA has something to say about this.

While writing the comment, I did think to myself, that NASA did a ton of prototypes to de-risk. They simulated the landing as close as they could possibly make it, on earth. So, probably not pure waterfall either. Maybe my comment was a bit too brusque in that regard.

Re: Verified Spec-Driven Development (VSDD)

#102
post #94

Earlier quoted context omitted.

Doesn't seem like too efficient process, no? Seems to me like investment in better quality of the output is exactly what is needed here, wouldn't you agree?

I feel they sit of on the opposite end of the OP here. One wants to write out specs to control the agent implementation to achieve a one shot execution. Other side says: let’s won’t waste time of humans writing anything. I’m personally torn. A lot of the spec talk and now here in combination with TDD etc feels like the pipe dreams of the mid 2000. There was this idea of the Architect role who writes UML and specs. An…

Actually, nobody said the spec needs to be written by humans.

My personal opinion: with today's LLMs, the spec should be steered by a human because its quality is proportional to result quality. Human interaction is much cheaper at that stage — it's all natural language that makes sense. Later, reasoning about the code itself will be harder.

In general, any non-trivial, valuable output must be based on some verification loop. A spec is just one way to express verification (natural language — a bit fuzzy, but still counts). Others are typecheckers, tests, and linters (especially when linter rules relate to correctness, not just cosmetics).

Personally, on non-trivial tasks, I see very good results with iterative, interactive, verifiable loops:

- Start with a task

- Write spec in e.g. SPEC.md → "ask question" until answer is "ok"/proceed

- Write implementation PLAN.md — topologically sorted list of steps, possibly with substeps → ask question

- For each step: implement, write tests, verify (step isn't done until tests pass, typecheck passes, etc.); update SPEC/PLAN as needed → ask question

- When done, convert SPEC.md and PLAN.md into PR description (summary) and discard

("Ask question" means an interactive prompt that appears for the user. Each step is gated by this prompt — it holds off further progress, giving you a chance to review and modify the result in small bits you can actually reason about.) The workflow: you accept all changes before confirming the next step. This way you get code deltas that make sense. You can review and understand them, and if something's wrong you can modify by hand (especially renames, which editors like VS Code handle nicely) or prompt for a change. The LLM is instructed to proceed only when the re-asked answer is "ok".

This works with systems like VSCode Copilot, not so much with CC cli.

I'm looking forward to an automated setup where the "human" is replaced by an "LLM judge" — I think you could already design a fairly efficient system like this, but for my work LLMs aren't quite there yet.

That said, there's an aspect that shouldn't be forgotten: this interactive approach keeps you in the driving seat and you know what's happening with the codebase, especially if you're running many of these loops per day. Fully automated solutions leave you outside the picture. You'll quickly get disconnected from what's going on — it'll feel more like a project run by another team where you kind of know what it does on the surface but have no idea how. IMO this is dangerous for long-term, sustainable development.

Re: Verified Spec-Driven Development (VSDD)

#104

Its an interesting direction if you see it under the umbrella of diminishing costs: You build a product once with vibe coding and a design/ product hat. Once you know what works you rebuild it 100% in a framework like this. You do this every time from scratch when the tech debt or the mismatch between architecture and needs are too big.

You could also use the same framework always - that's what I'm doing anyway. But you gotta remember that no matter how well you spec it, first iteration of the specs is going to suck anyway.

But you vibe-code it anyway and see what happens. You'll start noticing obvious issues that you can track back to something in spec.

Then you throw away the entire thing (the entire project!) and start from scratch. Repeat until you have something you like.

Incremental specing doesn't work though. You need a clean room approach with only important learnings from previous iterations. Otherwise agent will never pick a hard but correct path.

Re: Verified Spec-Driven Development (VSDD)

#105
Some of the ideas here are codified in a simpler way in my TDD framework, CodeLeash (https://codeleash.dev).

The key difference is CodeLeash puts guardrails outside the model, as Claude Code hooks. It can’t forget or skip steps, it gets forced through a process. There are bypasses but they raise alarms that the model is also forced to review.

Putting guardrails outside the model forces deterministic process following. With a sufficiently capable process (TDD is a powerful way to build software) you can really scale up coding agent usage.

Re: Verified Spec-Driven Development (VSDD)

#106

LLM-assisted development feels a lot like trend-driven development. When dealing with technique and heterogenous prompts and goals, it’s easy to gain somewhat of a gambler’s fallacy with respect to a particular technique. Spec-driven development feels pretty questionable to me. I’m sure it works fine for feature work that is predictable or has been done before, but then I wonder why you’d waste your time with it. Pri…

Or maybe people who like talking much more than they like code are now very excited about the possibility that talking has eaten software development. This is exactly backwards. For many tasks, formal languages are better, more real, more beautiful than English. No matter how many millions of tokens you have, you will never talk the formulas of Fermat, Euler, and Gauss into irrelevance. And the same is true of good c…

  > Or maybe people who like talking much more than they like code are now very excited about the possibility that talking has eaten software development.
This has been my experience. In my team, the most excited people appear to be those who spend most of their time in meetings.

Re: Verified Spec-Driven Development (VSDD)

#107
post #15
post #6

Everything in this post stems from the assumption that you already know what you're doing, which is probably true for things you've built before. But I hope we can agree that you can't spec out something you have no clue how to build, let alone write the tests before you've even explored the boundaries of the problem space. That's completely unreasonable. My second point is that this approach is fundamentally wrong f…

Nothing of what you write here matches my experience with AI. Specification is worth writing (and spending a lot more time on than implementation) because it's the part that you can still control, fully read, understand etc. Once it gets into the code, reviewing it will be a lot harder, and if you insist on reviewing everything it'll slow things down to your speed. > If the cost of writing code is approaching zero, t…

You claim to disagreeing with OP but you seem to be describing basically the same core loop of planning and execution.

Doing OODA faster has always been the key thing to creating high quality outcomes.

Re: Verified Spec-Driven Development (VSDD)

#108
post #100
post #67

Earlier quoted context omitted.

So, rollback and try again with the insight. AI makes it cheap to implement complex first drafts and iterations. I'm building a CRM system for my business; first time it took about 2 weeks to get a working prototype. V4 from scratch took about 5 hours.

Yeah, if done right. In my experience, such a reimplementation is often lossy, if tests don’t enforce presence of all features and nonfunctional requirements. Maybe the primary value of the early versions is building up the test system, allowing an ideal implementation with that in place. Or put this way: We’re brute forcing (nicer term: evolutionizing) the codebase to have a better structure. Evolutionary pressure (…

What matters ultimately is the system achieves your goals. The clearer you can be about that the less the implementation detail actually matters.

For example; do you care if the UI has a purple theme or a blue one? Or if it's React or Vur. If you do that's part of your goals, if not it doesn't entirely matter if V1 is Blue and React, but V4 ends up Purple and Vue.

Re: Verified Spec-Driven Development (VSDD)

#109
post #106

Earlier quoted context omitted.

Or maybe people who like talking much more than they like code are now very excited about the possibility that talking has eaten software development. This is exactly backwards. For many tasks, formal languages are better, more real, more beautiful than English. No matter how many millions of tokens you have, you will never talk the formulas of Fermat, Euler, and Gauss into irrelevance. And the same is true of good c…

> Or maybe people who like talking much more than they like code are now very excited about the possibility that talking has eaten software development. This has been my experience. In my team, the most excited people appear to be those who spend most of their time in meetings.

There was an early era when the AI revolution was led by us Java users who had to deal with boilerplate lol. Plus review 4000 lines of code a day, a lot of it boilerplate as well. This was also the era of "clean code" and we were looking at adapters connected to adapters for no reason besides DRY being a holy rule.

Re: Verified Spec-Driven Development (VSDD)

#110
post #54
post #15

Earlier quoted context omitted.

Nothing of what you write here matches my experience with AI. Specification is worth writing (and spending a lot more time on than implementation) because it's the part that you can still control, fully read, understand etc. Once it gets into the code, reviewing it will be a lot harder, and if you insist on reviewing everything it'll slow things down to your speed. > If the cost of writing code is approaching zero, t…

This. Waterfall never worked for a reason. Humans and agents both need to develop a first draft, then re-evaluate with the lessons learned and the structure that has evolved. It’s very very time consuming to plan a complex, working system up front. NASA has done it, for the moon landing. But we don’t have those resources, so we plan, build, evaluate, and repeat.

"Waterfall" was primarily a strawman that the agile salesman made up. Sure, it existed it some form but was not widely practiced.
Post reply on HN