181–190 of 278 posts
Earlier quoted context omitted.
You didn’t answer what does you 250k lines do? How much money does it make? How many users does it have?
what if you move from reviewing the code to reviewing the spec?
There’s probably a world where you could do that if the spec was written in a formal language with no ambiguity and there was a rigorous system for translating from spec to code sure.
In my view, Spec-Driven systems are doomed to fail. There's nothing that couples the english language specs you've written with the actual code and behaviour of the system - unless your agent is being insanely diligent and constantly checking if the entire system aligns with your specs. This has been solved already - automated testing. They encode behaviour of the system into executables which actually tell you if yo…
Earlier quoted context omitted.
Spec Driven Development is a curious term - it suggests it is a kind of, or at least in the tradition of, Test Driven Development but it goes in the opposite direction!
Don't understand this - you can go spec -> test -> implementation and establish the test loop. Bit like the v model of old, actually.
1. Specs are subject to bit-rot, there's no impetus to update them as behaviour changes - unless your agent workflow explicitly enforces a thorough review and update of the specs, and unless your agent is diligent with following it. Lots of trust required on your LLM here.
2. There's no way to systematically determine if the behaviour of your system matches the specs. Imagine a reasonable sized codebase - if there's a spec document for every feature, you're looking at quite a collection of specs. How many tokens need be burnt to ensure that these specs are always up to date as new features come in and behaviour changes?
3. Specs are written in English. They're ambiguous - they can absolutely serve the planning and design phases, but this ambiguity prevents meaningful behaviour assertions about the system as it grows.
Contrast that with tests:
1. They are executable and have the precision of code. They don't just describe behaviour of the system, they validate that the system follows that behaviour, without ambiguity.
2. They scale - it's completely reasonable to have extensive codebases have all (if not most) of their behaviour covered by tests.
3. Updating is enforcable - assuming you're using a CI pipeline, when tests break, they must be updated in order to continue.
4. You can systematically determine if the tests fully describe the behaviour (ie. is all the behaviour tested) via mutation testing. This will tell you with absolute certainty if code is tested or not - do the tests fully describe the system's behaviour.
That being said, I think it's very valuable to start with a planning stage, even to provide a spec, such that the correct behaviour gets encoded into tests, and then instantiated by the implementation. But in my view, specs are best used within the design stage, and if left in the codebase, treated only as historical info for what went into the development of the feature. Attempting to use them as the source of truth for the behaviour of the system is fraught.
And I guess finally, I think that insofar as any framework uses the specs as the source of truth for behaviour, they're going to run into alignment problems since maintaining specs doesn't scale.
In my view, Spec-Driven systems are doomed to fail. There's nothing that couples the english language specs you've written with the actual code and behaviour of the system - unless your agent is being insanely diligent and constantly checking if the entire system aligns with your specs. This has been solved already - automated testing. They encode behaviour of the system into executables which actually tell you if yo…
Its already quite debatable whether software developers should be called software engineers, but this is just ridiculous.
There are so many of these "meta" frameworks going around. I have yet to see one that proves in any meaningful way they improve anything. I have a hard time believing they accomplish anything other than burn tokens and poison the context window with too much information. What works best IME is keeping things simple, clear and only providing the essential information for the task at hand, and iterating in manageable s…
[flagged]
Earlier quoted context omitted.
Large codebase. Yeah man, I have a small business, trying to grow but not easy going up against Ticketmaster.
A ticketmaster competitor doesn’t sound like a huge technical challenge unless you’re operating at scale. So my first question would be why do you have a large codebase so with so few customers?
The spec-driven approach resonates. I've found that the quality of the initial context you feed to AI coding tools determines everything downstream. Vague specs produce vague code that needs constant correction. One pattern that's worked well for me: instead of writing specs manually, I extract structured architecture docs from existing systems (database schemas, API endpoints, workflow logic) and use those as the sp…