[dead]
A sufficiently detailed spec is code
221–230 of 376 posts
Re: A sufficiently detailed spec is code
#222> There is no world where you input a document lacking clarity and detail and get a coding agent to reliably fill in that missing clarity and detail That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. Code is the detail being filled in. Furthermore, LLMs are the ultimate detail fillers, because they are language interpolat…
[deleted]
Re: A sufficiently detailed spec is code
#223Earlier quoted context omitted.
> That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. LLMs can generate (relatively small amounts of) working code from relatively terse descriptions, but I don’t think they can do so _reliably_. They’re more reliable the shorter the code fragment and the more common the code, but they do break down for complex description…
Neither can humans, but the industry has decades of experience with how to instruct and guide human developer teams using specs.
Humans have the ability to retrospect, push back on a faulty spec, push back on an unclarified spec, do experiments, make judgement calls and build tools and processes to account for their own foibles.
Re: A sufficiently detailed spec is code
#224Yes and LLMs can iterate and help you refine that spec more quickly.
Re: A sufficiently detailed spec is code
#225Earlier quoted context omitted.
They do more than that. If you ask for ui with a button that button won't be upside down even if you didn't specify its orientation. Lots of the detail can be inferred from general human preferences, which are present in the LLMs' training data. This extends way beyond CS stuff like details of algorithm implementations.
Isn't "not being upsidedown" just one of the default properties of a button in whatever GUI toolkit you are using? I'd be worried if an LLM _did_ start setting all the possible button properties.
Putting LLMs on a pedestal is very much in vogue these days.
Re: A sufficiently detailed spec is code
#226A spec is an envelope that contains all programs that comply. Creating this spec is often going to be harder than writing a single compliant program. Since every invocation of an LLM may create a different program, just like people, we will see that the spec will leave much room for good and bad implementations, and highlight the imprecision in the spec. Once we start using a particular implementation it often become…
> A spec is an envelope that contains all programs that comply. Creating this spec is often going to be harder than writing a single compliant program.
This perfectly explains the feeling I had when, 20 years into my career, I had to start writing specs. I could never quite put my finger on why it was harder than coding. My greater familiarity with coding didn't seem a sufficient explanation.
When writing a line of spec, I had to consider how it might be interpreted in the context of different implementations of other lines of spec - combinatorial nightmare.
But code is just a spec as far as, say, a C compiler is concerned. The compiler is free to implement the assembly however it likes. Writing that spec is definitely easier than writing the assembly (Fred Brookes said this, so it must be true).
So why the difference?
Re: A sufficiently detailed spec is code
#227I agree with most of what the author is saying, but the slogan that "a sufficiently detailed spec is code" can be misunderstood as "a sufficiently detailed spec is a program". The statement is only true if you read "code" as "statement in a formal language". Here's a (sketch of a) specification for a compiler:
> For every specification satisfied by the input program, the output program satisfies the same specification.
This is not a program and it does not become a program once you fill in the holes. Making the statement precise clearly requires a formal language, but that language can work at a higher level of abstraction than a programming language. So yes, a specification can absolutely be simpler than a program that implements it.
Re: A sufficiently detailed spec is code
#228Good engineering has one loop. Hand coding and codegen are just different workers inside that loop.
Re: A sufficiently detailed spec is code
#229This is relatable.
I did a side project with a non-technical co-founder a year ago and every time he told me what he wanted, I made a list of like 9 or 10 logical contradictions in his requirements and I had to walk him through what he said with drawings of the UI so that he would understand. Some stuff he wanted me to do sounded good in his head but once you walk through the implementation details, the solution is extremely confusing for the user or it's downright physically impossible to do based on cost or computational resource constraints.
Sure, most people who launched a successful product basically stumbled onto the perfect idea by chance on the first attempt... But what about the 99% others who fell flat on their face! You are the 99% and so if you want to succeed by actual merit, instead of becoming a statistic, you have to think about all this stuff ahead of time. You have to simulate the product and business in detail in your mind and ask yourself honestly; is this realistic? Before you even draw your first wireframe. If you find anything wrong with it, anything wrong at all; it means the idea sucks.
It's like; this feature is too computationally and/or financially expensive to offer for free and not useful enough to warrant demanding payment from users... You shouldn't even waste your time with implementation; it's not going to work! The fundamental economics of the software which exists in your imagination aren't going to magically resolve themselves after implementing in reality.
Translating an idea to reality never resolves any known problems; it only adds more problems!
The fact is that most non-technical people only have a very vague idea of what they want. They operate in a kind of wishy washy, hand-wavy emotion-centric environment and they think they know what they're doing but they often don't.
Re: A sufficiently detailed spec is code
#230> A sufficiently detailed spec is code This is exactly the argument in Brooks' No Silver Bullet. I still believe that it holds. However, my observation is that many people don't really need that level of details. When one prompts an AI to "write me a to-do list app", what they really mean is that "write me a to-do list app that is better that I have imagined so far", which does not really require detailed spec.
In this case a chatbot is also unlikely to succeed in pleasing the user—and how could it?
Indeed, because how could that LLM know what I didn't like about all other to-do apps? It will generate a new one more or less the same as the old one, but the user will be happy because "yay it's mine". Maybe. Or maybe not.