Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A sufficiently detailed spec is code

haskellforall.com

Post reply on HN
Print view View original

61–70 of 376 posts

Re: A sufficiently detailed spec is code

#61

Post by ozozozd »

Panzerschrek wrote: post #40

I am developing my own programming language, but I have no specification written for it. When people tell me that I need a specification, I reply that I already have one - the source code of the language compiler.

You are not wrong. But, they are not wrong either.

I feel like if you’re designing a language, the activity of producing the spec, which involves the grammar etc., would allow you to design unencumbered by whether your design is easy to implement. Or whether it’s a good fit for the language you are implementing the compiler with.

The OP also correctly identifies that thoughtful design takes a back seat in favor of action when we start writing the code.

Re: A sufficiently detailed spec is code

#63

Post by sda2 »

anonzzzies wrote: post #43
rdevilla wrote: post #7

I think it's only a matter of time before people start trying to optimize model performance and token usage by creating their own more technical dialect of English (LLMSpeak or something). It will reduce both ambiguity and token usage by using a highly compressed vocabulary, where very precise concepts are packed into single words (monads are just monoids in the category of endofunctors, what's the problem?). Grammat…

It was mentioned somewhere else on hn today, but why do I care about token usage? I prompt AI day and night for coding and other stuff via claude code max 200 and mistral; haven't had issues for many months now.

it’s a measure of efficiency. one might not care about tokens until vendors jack up the price and running your own comparable model is infeasible.

Re: A sufficiently detailed spec is code

#64

Post by kenjackson »

Code is usually over specified. I recently used AI to build an app for some HS kids. It built what I spec’wd and it was great. Is it what I would’ve coded? Definitely not. In code I have to make a bunch of decisions that I don’t care about. And some of the decisions will seem important to some, but not to others. For example, it built a web page whereas I would’ve built a native app. I didn’t care either way and it doesn’t matter either way. But those sorts of things matter when coding and often don’t matter at all for the goal of the implementation.

Re: A sufficiently detailed spec is code

#65

Post by ms_menardi »

lmm wrote: post #48
hintymad wrote: post #28

> A sufficiently detailed spec is code This is exactly the argument in Brooks' No Silver Bullet. I still believe that it holds. However, my observation is that many people don't really need that level of details. When one prompts an AI to "write me a to-do list app", what they really mean is that "write me a to-do list app that is better that I have imagined so far", which does not really require detailed spec.

> When one prompts an AI to "write me a to-do list app", what they really mean is that "write me a to-do list app that is better that I have imagined so far", which does not really require detailed spec. If someone was making a serious request for a to-do list app, they presumably want it to do something different from or better than the dozens of to-do list apps that are already out there. Which would require them t…

It could be an issue of discoverability too. Maybe they just haven't found the to-do app that does what they want, and it's easier to just... make one from scratch.

Re: A sufficiently detailed spec is code

#66

Post by steve_adams_86 »

rdevilla wrote: post #7

I think it's only a matter of time before people start trying to optimize model performance and token usage by creating their own more technical dialect of English (LLMSpeak or something). It will reduce both ambiguity and token usage by using a highly compressed vocabulary, where very precise concepts are packed into single words (monads are just monoids in the category of endofunctors, what's the problem?). Grammat…

The thing is, doesn't the LLM need to be trained on this dialect, and if the training material we have is mostly ambiguous, how do we disambiguate it for the purpose of training?

In my mind this is solving different problems. We want it to parse out our intent from ambiguous semantics because that's how humans actually think and speak. The ones who think they don't are simply unaware of themselves.

If we create this terse and unambiguous language for LLMs, it seems likely to me that they would benefit most from using it with each other, not with humans. Further, they already kind of do this with programming languages which are, more or less, terse and unambiguous expression engines for working with computers. How would we meaningfully improve on this, with enough training data to do so?

I'm asking sincerely and not rhetorically because I'm under no illusion that I understand this or know any better.

Re: A sufficiently detailed spec is code

#67

Post by ozozozd »

macinjosh wrote: post #24

IMHO, LLMs are better at Python and SQL than Haskell because Python and SQL syntax mirrors more aspects of human language. Whereas Haskell syntax reads more like a math equation. These are Large _Language_ Models so naturally intelligence learned from non-code sources transfers better to more human like programming languages. Math equations assume the reader has context not included in the written down part for what…

They are not called Context-Sensitive Large Language Models though.

LLMs are very good at bash, which I’d argue doesn’t read like language or math.

Re: A sufficiently detailed spec is code

#68

Post by quotemstr »

No, a spec is not code. It's possible to describe simple behavior that's nevertheless difficult to implement. Consider, say,

  fn sin(x: f16) -> f16
There are only 64k different f16s. Easy enough to test them all. A given sin() is either correct or it's not.

Yet sin() here can have a large number of different implementations. The spec alone under-determines the actual code.

Re: A sufficiently detailed spec is code

#69

Post by baxtr »

mfabbri77 wrote: post #51
hintymad wrote: post #28

> A sufficiently detailed spec is code This is exactly the argument in Brooks' No Silver Bullet. I still believe that it holds. However, my observation is that many people don't really need that level of details. When one prompts an AI to "write me a to-do list app", what they really mean is that "write me a to-do list app that is better that I have imagined so far", which does not really require detailed spec.

Yes. This happens because the training data contains countless SotA "to-do" apps. This argument does not scale well to other types of software.

Isn’t most standard software these days a permutation of things already done before?

Re: A sufficiently detailed spec is code

#70

Post by ozozozd »

charcircuit wrote: post #6

This articles ignores that AI agents have intelligence which means that they can figure out unspecified parts of the spec on their own. There is a lot of the design of software that I don't care about and I'm fine letting AI pick a reasonable approach.

Here is an easy example:

Say you and I both wrote the same spec that under-specifies the same parts. But we both expect different behavior, and trust that LLM will make the _reasonable_ choices. Hint: “The choice that I would have made.”

Btw, by definition, when we under-specify we leave some decisions to the LLM unknowingly.

And absent our looks or age as input, the LLM will make some _reasonable_ choices based on our spec. But will those choices be closer to yours or mine? Assuming it won’t be neither.

Post reply on HN