A sufficiently detailed spec is code • Hacker News Board

Re: A sufficiently detailed spec is code

#321

Post by abustamam » Thu, Mar 19, 2026, 4:57 PM UTC

anonzzzies wrote: post #43

rdevilla wrote: post #7

I think it's only a matter of time before people start trying to optimize model performance and token usage by creating their own more technical dialect of English (LLMSpeak or something). It will reduce both ambiguity and token usage by using a highly compressed vocabulary, where very precise concepts are packed into single words (monads are just monoids in the category of endofunctors, what's the problem?). Grammat…

It was mentioned somewhere else on hn today, but why do I care about token usage? I prompt AI day and night for coding and other stuff via claude code max 200 and mistral; haven't had issues for many months now.

You may not but many people do. My boss routinely runs over his quota of Claude max.

Re: A sufficiently detailed spec is code

#322

Post by abustamam » Thu, Mar 19, 2026, 5:01 PM UTC

stanac wrote: post #177

jumploops wrote: post #160

In my experience with “agentic engineering” the spec docs are often longer than the code itself. Natural language is imperfect, code is exact. The goal of specs is largely to maintain desired functionality over many iterations, something that pure code handles poorly. I’ve tried inline comments, tests, etc. but what works best is waterfall-style design docs that act as a second source of truth to the running code. Us…

> The goal of specs is largely to maintain desired functionality over many iterations, something that pure code handles poorly. IMHO this could be achieved with large set of tests, but the problem is if you prompt an agent to fix tests, you can't be sure it won't "fix the test". Or implement something just to make the test pass without looking at a larger picture.

I find myself babysitting agent-derived tests unless I specifically say what the in variants and edge cases are. Sometimes I'll ask it if I missed anything and it'll be helpful. But I have to be proactive.

Re: A sufficiently detailed spec is code

#323

Post by lowbloodsugar » Thu, Mar 19, 2026, 5:07 PM UTC

bad_username wrote: post #145

> There is no world where you input a document lacking clarity and detail and get a coding agent to reliably fill in that missing clarity and detail That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. Code is the detail being filled in. Furthermore, LLMs are the ultimate detail fillers, because they are language interpolat…

Isn’t this no different than a compiler? I don’t specify which registers to use in C or Java, and yet the code runs!

An awful lot of time has been put into the compiler to know which registers to use and how to juggle them. Is an LLM any different in its behavior (albeit different in how it was trained)? If not, then specs are just an even higher level programming language.

I think the difference is that, with a C compiler, when it gets it wrong I’ll have some terrible performance impact, it when the LLM gets it wrong, it will do something nobody wanted, like delete someone’s account or debit one account without crediting another.

Re: A sufficiently detailed spec is code

#324

Post by amtamt » Thu, Mar 19, 2026, 5:08 PM UTC

IanCal wrote: post #204

amtamt wrote: post #161

> On two occasions I have been asked [by members of Parliament], 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. I guess many of us quality for british parliament.

I cannot shake the feeling that with that quote the people were trying to point out a problem with something he’d said. “With this machine, all the results will be correct, no more errors in log tables” “And what if people put the wrong figures in?” (Hint - we’d still have the wrong results) Babbage walks away thinking them an idiot, they walk away thinking Babbage hasn’t considered anything outside of the machine it…

Even two wrong, but countering, inputs can sometimes provide right answer, so the rational above has some merit.

But if I interpret the question with line of thinking "should I anticipate right/ full answers despite incorrect/ incomplete inputs?" I think Baggage was pointing out the problem in the logic why such questions should arise.

I would expect the question to be phrased "under what circumstances the machine with provide wrong outputs?", and would have hoped for Babbage (or may be anyone) explaining many ways how things could go wrong.

Re: A sufficiently detailed spec is code

#325

Post by Shebanator » Thu, Mar 19, 2026, 5:13 PM UTC

agentultra wrote: post #276

bad_username wrote: post #145

> There is no world where you input a document lacking clarity and detail and get a coding agent to reliably fill in that missing clarity and detail That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. Code is the detail being filled in. Furthermore, LLMs are the ultimate detail fillers, because they are language interpolat…

We do have such detailed specifications. But they are written in a language with a narrow interface. It’s a technique called, “program synthesis,” and you can find an example of such a language called, Synquid . It might be illuminating to see what a mathematically precise specification can and cannot do when it comes to generating programs. A major challenge in formal methods is proving that the program implements t…

This works well for problems that are purely algorithmic in nature. But problems often have solutions that don't fall into those categories, especially in UI/UX. When people tell me that LLMs can solve anything with a sufficiently details spec, I ask them to produce such a spec for Adobe Photoshop.

Re: A sufficiently detailed spec is code

#326

Post by ppeetteerr » Thu, Mar 19, 2026, 5:19 PM UTC

Too much of code is data transformation. input -> sanitation -> db -> consumer -> api -> client. Business logic defines the shape of that data and some service-level rules but the majority is just shoveling data.

Re: A sufficiently detailed spec is code

#327

Post by bombcar » Thu, Mar 19, 2026, 5:23 PM UTC

Sufficiently detailed code is the spec.

https://xkcd.com/1172/

Re: A sufficiently detailed spec is code

#328

Post by mohamedkoubaa » Thu, Mar 19, 2026, 5:26 PM UTC

This holds in the opposite direction. Why should an LLM rtfm when it could just as well read the implementation? Of course, you might need a copy of the code in your workspace that it can index, and that usually isn't a problem

Re: A sufficiently detailed spec is code

#329

Post by reedlaw » Thu, Mar 19, 2026, 5:30 PM UTC

mike_hearn wrote: post #192

Gabriel439 wrote: post #109

Earlier quoted context omitted.

Author here: it's not even clear that agents can reliably permute their training data (I'm not saying that it's impossible or never happens but that it's not something we can take for granted as a reliable feature of agentic coding). As I mentioned in one of the footnotes in the post: > People often tell me "you would get better results if you generated code in a more mainstream language rather than Haskell" to which…

Your argument is far too dependent on observations made about the model's ability with Haskell, which is irrelevant. The concepts in Haskell are totally different to almost any other language - you can't easily "generalize" from an imperative strict language like basically everything people really use to a lazy pure FP language that uses monads for IO like Haskell. The underlying concepts themselves are different and…

Why is Haskell irrelevant to the argument that LLMs can't reliably permute programming knowledge from one language to another? In fact, the purity of the language and dearth of training data seems like the perfect test case to see whether concepts found in more mainstream languages are actually understood.

Re: A sufficiently detailed spec is code

#330

Post by derrak » Thu, Mar 19, 2026, 5:46 PM UTC

angry_octet wrote: post #162

A spec is an envelope that contains all programs that comply. Creating this spec is often going to be harder than writing a single compliant program. Since every invocation of an LLM may create a different program, just like people, we will see that the spec will leave much room for good and bad implementations, and highlight the imprecision in the spec. Once we start using a particular implementation it often become…

> Creating this spec is often going to be harder than writing a single compliant program.

I’m not sure if I agree. A good spec will specify what the program is supposed to do rather than how the program should do that thing. It should be easier to specify what rather than how. Moreover, it should be standard practice to express what before you start writing how.

I see some people pushing back on this by saying two programs that satisfy the same spec might have different performance or security properties. That is correct and if you care about those things, you should specify them. Writing down these properties, e.g., “the program is O(n) where n is blah” should be much easier than implementing a non-trivial linear time algorithm founded in deep ideas.