Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A sufficiently detailed spec is code

haskellforall.com

Post reply on HN
Print view View original

201–210 of 376 posts

Re: A sufficiently detailed spec is code

#201

Post by rdevilla »

mike_hearn wrote: post #196
grey-area wrote: post #132

Earlier quoted context omitted.

Have you just reinvented programming languages and reinforced the author's point? Setting aside the problem of training, why bother prompting if you’re going to specify things so tightly that it resembles code?

Programming languages admit only unambiguous text. What he's proposing is more like EARS, Gherkin or Planguage.

Not necessarily. I was intending it as a thought experiment illustrating why some kind of formal language (whether that mean technical jargon, unambiguous syntax, unambiguous semantics, conlangs, specification languages, or some combination thereof) will eventually arise from natural language - as it has countless times in the past, within mathematics (as referenced in TFA) and elsewhere. Gherkin is kind of nice though.

Re: A sufficiently detailed spec is code

#202

Post by abcde666777 »

charcircuit wrote: post #154
abcde666777 wrote: post #16

Earlier quoted context omitted.

These algorithms don't have intelligence, they just regurgitate human intelligence that was in their training data. That also goes the other way - they can't produce intelligence that wasn't represented in their training input.

Firstly, it doesn't really matter if they can produce novel designs or not. 99% of what is being done is not novel. It is manipulating data in ways computers have been manipulating data for decades. The design of what is implemented is also going to be derivative of what already exists in the world too. Being too novel makes for a bad product since users will not easily understand how to use it and adapt their existi…

Chess AI isn't trained in the same way. Things like alpha zero partly worked by playing themselves recursively, meaning they actually did generate novel data in the process.

That was partly possible because chess is a constrained domain: rigid rules and board states.

But LLM land is not like that. LLM land was trained on pre-existing text written by humans. They do discover patterns within said data but the point stands that the data and patterns within are not actually novel.

Re: A sufficiently detailed spec is code

#204

Post by IanCal »

amtamt wrote: post #161

> On two occasions I have been asked [by members of Parliament], 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. I guess many of us quality for british parliament.

I cannot shake the feeling that with that quote the people were trying to point out a problem with something he’d said.

“With this machine, all the results will be correct, no more errors in log tables”

“And what if people put the wrong figures in?” (Hint - we’d still have the wrong results)

Babbage walks away thinking them an idiot, they walk away thinking Babbage hasn’t considered anything outside of the machine itself.

Re: A sufficiently detailed spec is code

#205

Post by roysting »

bad_username wrote: post #145

> There is no world where you input a document lacking clarity and detail and get a coding agent to reliably fill in that missing clarity and detail That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. Code is the detail being filled in. Furthermore, LLMs are the ultimate detail fillers, because they are language interpolat…

In get the sense that what you are responding to and even many comments to yours are expressing a kind of coping with the current dynamic, only exacerbated by the rather elitist and egoistic mentality that people in tech have had for a very long time now; i.e., they are falling…being pushed from Mt Olympus and there is A LOT of anxious rationalization going on.

Not a mere 5 years ago even tech people were chortling down their upturned noses that people were complaining that their jobs were being “taken”, and now that the turns have tabled, there is a bunch of denial, anger, and grief going on, maybe even some depression as many of the recently unemployed realize the current state of things.

It’s all easy to deride the inferiority of AI when you’re employed in a job doing things as you had been all your career, thinking you cannot be replaced… until you find yourself on the other side of the turn that has tabled.

Re: A sufficiently detailed spec is code

#206

Post by charcircuit »

bigstrat2003 wrote: post #50
systemsweird wrote: post #10

Earlier quoted context omitted.

Exactly. The real speed up from AI will come when we can under specify a system and the AI uses its intelligence to make good choices on the parts we left out. If you have to spec something out with zero ambiguity you’re basically just coding in English. I suspect current ideas around formal/detailed spec driven development will be abandoned in a couple years when models are significantly better.

> in a couple years when models are significantly better. They aren't significantly better now than a couple of years ago. So it doesn't seem likely they will be significantly better in a couple of years than they are now.

A couple years ago we didn't even have thinking. AI could barely complete a line of code that you working on and now they are capable of long running tasks like building an entire C compiler.

Re: A sufficiently detailed spec is code

#207

Post by billynomates »

amtamt wrote: post #161

> On two occasions I have been asked [by members of Parliament], 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. I guess many of us quality for british parliament.

Maybe the joke was on Babbage. Possible that they were asking about trust and over-reliance, which turned out to be the real problem.

Re: A sufficiently detailed spec is code

#208

Post by dxdm »

mike_hearn wrote: post #190
Someone wrote: post #157

Earlier quoted context omitted.

> That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. LLMs can generate (relatively small amounts of) working code from relatively terse descriptions, but I don’t think they can do so _reliably_. They’re more reliable the shorter the code fragment and the more common the code, but they do break down for complex description…

Neither can humans, but the industry has decades of experience with how to instruct and guide human developer teams using specs.

Usually, you don't want your developers to be coding monkeys, for good results. You need the human developer in the loop to even define the spec, maybe contributing ideas, but at the very least asking questions about "what happens when..." and "have you thought about...".

In fact, this is a huge chunk of the value a developer brings to the table.

Re: A sufficiently detailed spec is code

#209

Post by charcircuit »

abcde666777 wrote: post #202
charcircuit wrote: post #154

Earlier quoted context omitted.

Firstly, it doesn't really matter if they can produce novel designs or not. 99% of what is being done is not novel. It is manipulating data in ways computers have been manipulating data for decades. The design of what is implemented is also going to be derivative of what already exists in the world too. Being too novel makes for a bad product since users will not easily understand how to use it and adapt their existi…

Chess AI isn't trained in the same way. Things like alpha zero partly worked by playing themselves recursively, meaning they actually did generate novel data in the process. That was partly possible because chess is a constrained domain: rigid rules and board states. But LLM land is not like that. LLM land was trained on pre-existing text written by humans. They do discover patterns within said data but the point sta…

>LLM land was trained on pre-existing text written by humans.

Some of the pretraining. Other pretraining is on text written by AI. Human training data is only but a subset of what these models train on. There is a ton of synthetic training data now.

Re: A sufficiently detailed spec is code

#210

Post by rdevilla »

roysting wrote: post #205
bad_username wrote: post #145

> There is no world where you input a document lacking clarity and detail and get a coding agent to reliably fill in that missing clarity and detail That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. Code is the detail being filled in. Furthermore, LLMs are the ultimate detail fillers, because they are language interpolat…

In get the sense that what you are responding to and even many comments to yours are expressing a kind of coping with the current dynamic, only exacerbated by the rather elitist and egoistic mentality that people in tech have had for a very long time now; i.e., they are falling…being pushed from Mt Olympus and there is A LOT of anxious rationalization going on. Not a mere 5 years ago even tech people were chortling d…

I can't help but imagine that this is how some people felt about doctors once webmd came out.

It's some nice rhetoric, but you're not actually saying much.

Post reply on HN