Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A sufficiently detailed spec is code

haskellforall.com

Post reply on HN
Print view View original

271–280 of 376 posts

Re: A sufficiently detailed spec is code

#271

Post by hnthrow0287345 »

bad_username wrote: post #145

> There is no world where you input a document lacking clarity and detail and get a coding agent to reliably fill in that missing clarity and detail That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. Code is the detail being filled in. Furthermore, LLMs are the ultimate detail fillers, because they are language interpolat…

It means for most specs, you can just use an average solution, or a most-popular solution.

I'm absolutely on board with that. We probably need less weird and outlier decisions in designs for something that is a boring ass business website.

Re: A sufficiently detailed spec is code

#272

Post by ithkuil »

nextaccountic wrote: post #140
kstenerud wrote: post #130

Earlier quoted context omitted.

Human language is already very efficient for conveying the ideas we have. Some languages are more efficient at conveying certain concepts, but all are able to handle the 90% case. I'd expect any attempts to build a "technical dialect of English" to go about as well as Esperanto.

We already speak in a "technical dialect of English". All we need is some jargon to talk about technical things. (Lawyers have their own jargon too, also chemists, etc) Some languages don't have this kind of vocabulary, because there aren't enough speakers that deal with technical things in a given area (and those that do, use another language to communicate)

For me the proof that we do indeed have and use a technical dialect of English in this field lies in the simple observation that no matter how much praise I get at work for how good my English, that doesn't map at all with my ability (or lack thereof) to converse fluently with the random Joe on the street of an English speaking country

Re: A sufficiently detailed spec is code

#273

Post by gusmd »

dxdm wrote: post #208
mike_hearn wrote: post #190

Earlier quoted context omitted.

Neither can humans, but the industry has decades of experience with how to instruct and guide human developer teams using specs.

Usually, you don't want your developers to be coding monkeys, for good results. You need the human developer in the loop to even define the spec, maybe contributing ideas, but at the very least asking questions about "what happens when..." and "have you thought about...". In fact, this is a huge chunk of the value a developer brings to the table.

And this is usually one of the defining traits of a senior engineer. They understand the tech and its limitations, and thus are able to look around corners, ask good questions, and, overall, provide quality product input.

Re: A sufficiently detailed spec is code

#274

Post by MoreQARespect »

pablobaz wrote: post #231
MoreQARespect wrote: post #223

Earlier quoted context omitted.

Humans have the ability to retrospect, push back on a faulty spec, push back on an unclarified spec, do experiments, make judgement calls and build tools and processes to account for their own foibles.

which bits of this do you think llm based agents can't do?

any of them.

Re: A sufficiently detailed spec is code

#275

Post by causalityltd »

causalityltd wrote: post #59

The cognitive dissonance comes from the tension between the-spec-as-management-artifact vs the-spec-as-engineering-artifact. Author is right that advocates are selling the first but second is the only one which works. For a manager, the spec exists in order to create a delgation ticket, something you assign to someone and done. But for a builder, it exists as a thinking tool that evolves with the code to sharpen the…

Also, however much you manage the project, eventually you do need to actually "build". You can't deliver on hype alone. Or maybe you can, but only for some news cycles or VC meetings. The user will eventually need the promised product.

Re: A sufficiently detailed spec is code

#276

Post by agentultra »

bad_username wrote: post #145

> There is no world where you input a document lacking clarity and detail and get a coding agent to reliably fill in that missing clarity and detail That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. Code is the detail being filled in. Furthermore, LLMs are the ultimate detail fillers, because they are language interpolat…

We do have such detailed specifications. But they are written in a language with a narrow interface. It’s a technique called, “program synthesis,” and you can find an example of such a language called, Synquid.

It might be illuminating to see what a mathematically precise specification can and cannot do when it comes to generating programs. A major challenge in formal methods is proving that the program implements the specification faithfully, known as the specification gap. If you have a very high level and flexible specification language, such as TLA+, there is a lot of work to do to verify that the program you write meets the specification you wrote. For something like Synquid that is closer to the code there are more constraints on expressivity.

The point is that spoken language is not sufficiently precise to define a program.

Just because an LLM can fill in plausible details where sufficient detail is lacking doesn’t indicate that it’s solving the specification gap. If the program happens to implement the specification faithfully you got lucky. You still don’t actually know that’s true until you verify it.

It’s different with a narrow interface though: you can be very precise and very abstract with a good mathematical system for expressing specifications. It’s a lot more work and requires more training to do than filling in a markdown file and trying to coax the algorithm into outputting what you want through prose and fiction.

Re: A sufficiently detailed spec is code

#277

Post by abdulhaq »

bad_username wrote: post #145

> There is no world where you input a document lacking clarity and detail and get a coding agent to reliably fill in that missing clarity and detail That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. Code is the detail being filled in. Furthermore, LLMs are the ultimate detail fillers, because they are language interpolat…

> Sometimes the interpolated detail is wrong

you just (correctly) negated your own claim

Re: A sufficiently detailed spec is code

#278

Post by ngriffiths »

I think in an ideal world, programming languages[1] would allow you to express key behaviors, interfaces, states, etc very precisely and sharpen your own understanding while writing, but importantly, also leave unimportant behaviors, structures, etc unspecified, using reasonable defaults and similar stuff.

I sort of see the success of coding agents as not just a sign that people get tricked by slop, but mainly a sign that today's languages and frameworks just aren't that good. And now it's easier to say to the LLM, "I literally don't care how the UI is organized as long as everything is visible" than to try to express that in code.

[1]: I mean this in the vaguest sense, so including DSLs/frameworks/interfaces offered by libraries too

Re: A sufficiently detailed spec is code

#279

Post by IsTom »

lukevp wrote: post #186
ozlikethewizard wrote: post #156

Earlier quoted context omitted.

The hard bit of programming has never been knowing the symbols to tell the computer what to do. It is more difficult to use a completely unknown language, sure, but the paradigms and problem solving approaches are identical and thats the actual work, not writing the correct words.

Saying that the paradigms of Python and Haskell are the same makes it sound like you don’t know either or both of those languages. They are not just syntactically different. The paradigms literally are different. Python is a high level duck typed oo scripting language and Haskell is a non-oo strongly typed functional programming language. They’re extremely far apart.

They are different, but on some fundamental level when you're writing code you're expressing an idea and it is still the same. The same way photograph and drawing of a cat are obviously different and they're made in vastly different ways, but still they're representations of a cat. It's all lambda calculus, turing machines, ram machines, combinator logic, posts' dominoes etc., etc. in the end.

Re: A sufficiently detailed spec is code

#280

Post by jes5199 »

Someone wrote: post #157
bad_username wrote: post #145

> There is no world where you input a document lacking clarity and detail and get a coding agent to reliably fill in that missing clarity and detail That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. Code is the detail being filled in. Furthermore, LLMs are the ultimate detail fillers, because they are language interpolat…

> That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. LLMs can generate (relatively small amounts of) working code from relatively terse descriptions, but I don’t think they can do so _reliably_. They’re more reliable the shorter the code fragment and the more common the code, but they do break down for complex description…

anything can be reliable if you have good tests
Post reply on HN