I’d say it differently - that without code the spec is insufficient. Maybe you don’t need a full program as the spec but without some code, you’re left trying to be precise in natural language and that’s not what they are good for.
A sufficiently detailed spec is code
241–250 of 376 posts
Re: A sufficiently detailed spec is code
#242I don't agree. The code is much more than the spec. In fact, the typical project code is 90% scaffolding and infrastructure code to put together and in fact contains implementation details specific to the framework you use. And only 10% or less is actual "business logic". The spec doesn't have to deal with language, framework details, so by definition spec is the minimum amount of text necessary to express the busine…
I agree. Simple example: if you have a program written with MySQL as the backend, in most cases you could switch to e.g. Postgres and still meet the spec in every meaningful sense of spec. Will the code change? More than just a find/replace of "MySQL" to "Postgres"? Of course.
Re: A sufficiently detailed spec is code
#243Earlier quoted context omitted.
Humans have the ability to retrospect, push back on a faulty spec, push back on an unclarified spec, do experiments, make judgement calls and build tools and processes to account for their own foibles.
Humans also have the ability to introspect . Ultimately, (nearly) every software project is intended to provide a service to humans, and most humans are similar in most ways: "what would I want it to do?" is a surprisingly-reliable heuristic for dealing with ambiguity, especially if you know where you should and shouldn't expect it to be valid. The best LLMs can manage is "what's statistically-plausible behaviour for…
Most companies I've worked for have had 'know the customer' events so that developers learn what the customers really do and in turn even if we are not in their domain we have a good idea what they care about.
Re: A sufficiently detailed spec is code
#244I think it's only a matter of time before people start trying to optimize model performance and token usage by creating their own more technical dialect of English (LLMSpeak or something). It will reduce both ambiguity and token usage by using a highly compressed vocabulary, where very precise concepts are packed into single words (monads are just monoids in the category of endofunctors, what's the problem?). Grammat…
Let's use a non-ambiguous language for this. May I suggest Lojban [1][2]? [1] https://en.wikipedia.org/wiki/Lojban [2] Someone speaking it: https://www.youtube.com/watch?v=lxQjwbUiM9w
Lojban allows you to be vastly more semantically ambiguous than English while still speaking technically correctly. A single predicate-word (“gismu”) is a valid utterance. For example, saying “tanxe” is so vague or context-dependent enough as to be hard to translate: “something unspecified is a box, at an unspecified time or place, and it may or may not even exist”. Language will not save you. Or if it will, we already have them in the form of programming languages.
Re: A sufficiently detailed spec is code
#245Earlier quoted context omitted.
They do more than that. If you ask for ui with a button that button won't be upside down even if you didn't specify its orientation. Lots of the detail can be inferred from general human preferences, which are present in the LLMs' training data. This extends way beyond CS stuff like details of algorithm implementations.
Isn't "not being upsidedown" just one of the default properties of a button in whatever GUI toolkit you are using? I'd be worried if an LLM _did_ start setting all the possible button properties.
If you ask for increase and decrease buttons they will put the right icons on it (not words) and lay them out right.
Re: A sufficiently detailed spec is code
#246> There is no world where you input a document lacking clarity and detail and get a coding agent to reliably fill in that missing clarity and detail That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. Code is the detail being filled in. Furthermore, LLMs are the ultimate detail fillers, because they are language interpolat…
In get the sense that what you are responding to and even many comments to yours are expressing a kind of coping with the current dynamic, only exacerbated by the rather elitist and egoistic mentality that people in tech have had for a very long time now; i.e., they are falling…being pushed from Mt Olympus and there is A LOT of anxious rationalization going on. Not a mere 5 years ago even tech people were chortling d…
As you can see by downvotes and comments, they still don't get it.
LLMs make developers more efficient. That much is obvious to anyone who isn't blinded by fear.
But people will respond "but you still need developers!" True. You don't need nearly as many, though. In fact, with an LLM in their hands, the poor performers are more of a liability than ever. They'll be let go first.
But even the "smart" developers will be subsumed, as vastly more efficient companies outcompete the ones where they work.
Companies with slop-tolerant architectures will take over every industry. They'll have humans working there. But not many.
Re: A sufficiently detailed spec is code
#247I’ve noticed that the so-called industry is always fixated on the most crappy language. First it was JavaScript, then Python, now English. They have the common problem of being too flexible and ambiguous, causing bugs that are incredibly hard to find. Yet somehow they always become the most popular, and there’s always a cult about it, mostly consist of people who aren’t real computer scientists.
“This time is different” are the famous last words.
Re: A sufficiently detailed spec is code
#248Earlier quoted context omitted.
> That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. LLMs can generate (relatively small amounts of) working code from relatively terse descriptions, but I don’t think they can do so _reliably_. They’re more reliable the shorter the code fragment and the more common the code, but they do break down for complex description…
Neither can humans, but the industry has decades of experience with how to instruct and guide human developer teams using specs.
When LLMs generate an appropriate program from ambiguous requirements, they do this because the requirements happen to match something similar that has been done previously elsewhere.
There is a huge amount of programming work that consists in reinventing the wheel, i.e. in redoing something very similar to programs that have been written thousands of times before.
For this kind of work LLMs can greatly improve productivity, even if they are not much better than if you would be allowed to search, copy and paste from the programs on which the LLM has been trained. The advantage of an LLM is the automation of the search/copy/paste actions, and even more than this, the removal of the copyrights from the original programs. The copyright laws are what has resulted in huge amounts of superfluous programming work, which is necessary even when there are open-source solutions, but the employer of the programmer wants to "own the IP".
On the other hand, for really novel applications, or for old applications where you want to obtain better performance than anyone has gotten before, providing an ambiguous prompt to an LLM will get you nowhere.
Re: A sufficiently detailed spec is code
#249Re: A sufficiently detailed spec is code
#250> A sufficiently detailed spec is code This is exactly the argument in Brooks' No Silver Bullet. I still believe that it holds. However, my observation is that many people don't really need that level of details. When one prompts an AI to "write me a to-do list app", what they really mean is that "write me a to-do list app that is better that I have imagined so far", which does not really require detailed spec.
Everyone at least heard stories of people who just want that button 5px to the right or to the left and next meeting they want it in bottom corner - whereas it doesn’t make functionally any difference. But that’s most of the time is not that they want it from objective technical reasons. They want it because they want to see if they can push you. They do it „because they can”. They do it because later they can renego…
And that’s why you make your code modular and impervious to such change. And I don’t mind if they want to spend their money by making me do stuff that’s not important.