Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dijkstra On the foolishness of "natural language programming"

cs.utexas.edu

Post reply on HN
Print view View original

131–140 of 281 posts

Re: Dijkstra On the foolishness of "natural language programming"

#131

Post by soraminazuki »

soulofmischief wrote: post #125
soraminazuki wrote: post #120

Earlier quoted context omitted.

LLMs produce deterministic results ? Now, that 's a big [citation needed]. Where can I find the specs? Edit: This is assuming by "deterministic," you mean the same thing I said about programming language implementations being "controllable, reproducible, and well-defined." If you mean it produces random but same results for the same inputs, then you haven't made any meaningful points.

I'd recommend learning how transformers work, and the concept of temperature. I don't think I need to cite information that is broadly and readily available, but here: https://medium.com/google-cloud/is-a-zero-temperature-determ... I also qualified the requirement of needing the same hardware, due to FP shenanigans. I could further clarify that you need the same stack (pytorch, tensorflow, etc)

This gcc script that I created below is just as "deterministic" as an LLM. It produces the same result every time. Doesn't make it useful though.

    echo '#!/usr/bin/env bash' > gcc
    echo 'cat > gcc
    openssl rand -base64 100 >> gcc
    echo 'EOF' >> gcc
    chmod +x gcc
Also, how transformers work is not a spec of the LLM that anyone can use to learn how LLM produces code. It's no gcc source code.

Re: Dijkstra On the foolishness of "natural language programming"

#132

Post by fourside »

eightysixfour wrote: post #127
01100011 wrote: post #24

People are sticking up for LLMs here and that's cool. I wonder, what if you did the opposite? Take a project of moderate complexity and convert it from code back to natural language using your favorite LLM. Does it provide you with a reasonable description of the behavior and requirements encoded in the source code without losing enough detail to recreate the program? Do you find the resulting natural language descri…

Language can carry tremendous amounts of context. For example: > I want a modern navigation app for driving which lets me select intersections that I never want to be routed through. That sentence is low complexity but encodes a massive amount of information. You are probably thinking of a million implementation details that you need to get from that sentence to an actual working app but the opportunity is there, the…

You call it context or information but I call it assumptions. There are a ton assumptions in that sentence that an LLM will need to make in order to take that and turn it into a v1. I’m not sure what resulting app you’d get but if you did get a useful starting point, I’d wager the fact that you chose a variation of an existing type of app helped a lot. That is useful, but I’m not sure this is universally useful.

Re: Dijkstra On the foolishness of "natural language programming"

#133

Post by layer8 »

card_zero wrote: post #58
fluidcruft wrote: post #54

Earlier quoted context omitted.

I'm not so sure it's about precision rather than working memory. My presumption is people struggle to understand sufficiently large prose versions for the same reason a LLM would struggle working with larger prose versions: people have limited working memory. The time needed to reload info from prose is significant. People reading large text works will start highlighting and taking notes and inventing shorthand forms…

So is more compact better? Does K&R's *d++ = *s++; get a pass now?

When I first read the K&R book, that syntax made perfectly sense. They are building up to it through a few chapters, if I remember correctly.

What has changed is that nowadays most developers aren't doing low-level programming anymore, where the building blocks of that expression (or the expression itself) would be common idioms.

Re: Dijkstra On the foolishness of "natural language programming"

#134

Post by truculent »

Any sufficiently advanced method of programming will start to look less like natural language and more like a programming language.

If you still don’t want to do programming, then you need some way to instruct or direct the intelligence that _will_ do the programming.

And any sufficiently advanced method of instruction will look less like natural language, and more like an education.

Re: Dijkstra On the foolishness of "natural language programming"

#135

Post by musesum »

Personal context is tacit experience. Cultural context is aggregate.

Wittgenstein stated that the differences between personal and cultural language makes it impossible to agree on anything foundational to philosophy.

Godel did something similar to discrete structures by hacking self reference -- a kind of recursive self reference with no exit state.

I think pair programming with an LLM is interesting. You get to compare personal context with aggregate context. Plus, the external feedback helps me break out of excessive personal self reference.

Re: Dijkstra On the foolishness of "natural language programming"

#136

Post by artifaxx »

azernik wrote: post #85
sim7c00 wrote: post #53

Earlier quoted context omitted.

you guys are not wrong. explain any semi complez program, you will instantly resort to diagrams, tables, flow charts etc. etc. ofcourse, you can get your LLM to be bit evil in its replies, to help you truly. rather than to spoon feed you an unhealthy diet. i forbid my LLM to send me code and tell it to be harsh to me if i ask stupid things. stupid as in, lazy questions. send me the link to the manual/specs with an RT…

If you don't mind sharing - what's the specific prompt you use to get this to happen, and which LLM do you use it with?

I can share a similar approach I'm finding beneficial. I add "Be direct and brutally honest in your feedback. Identify assumptions and cognitive biases to correct for." (I also add a compendium of cognitive biases and examples to the knowledge I give the LLM.

Re: Dijkstra On the foolishness of "natural language programming"

#137

Post by layer8 »

fluidcruft wrote: post #54
01100011 wrote: post #24

People are sticking up for LLMs here and that's cool. I wonder, what if you did the opposite? Take a project of moderate complexity and convert it from code back to natural language using your favorite LLM. Does it provide you with a reasonable description of the behavior and requirements encoded in the source code without losing enough detail to recreate the program? Do you find the resulting natural language descri…

I'm not so sure it's about precision rather than working memory. My presumption is people struggle to understand sufficiently large prose versions for the same reason a LLM would struggle working with larger prose versions: people have limited working memory. The time needed to reload info from prose is significant. People reading large text works will start highlighting and taking notes and inventing shorthand forms…

Another important difference is reproducibility. With the same program code, you are getting the same program. With the same natural-language specification, you will presumably get a different thing each time you run it through the "interpreter". There is a middle ground, in the sense that a program has implementation details that aren't externally observable. Still, making the observable behavior 100% deterministic by mere natural-language description doesn't seem a realistic prospect.

Re: Dijkstra On the foolishness of "natural language programming"

#138

Post by squeaky-clean »

ratorx wrote: post #124
tim333 wrote: post #118

Earlier quoted context omitted.

> prefer the coded format. Is is compact... On the other hand "a folder that syncs files between devices and a server" is probably a lot more compact than the code behind Dropbox. I guess you can have both in parallel - prompts and code.

Let’s say that all of the ambiguities are automatically resolved in a reasonable way. This is still not enough to let 2 different computers running two different LLMs to produce compatible code right? And no guarantee of compatibility as you refine it more etc. And if you get into the business of specifying the format/protocol, suddenly you have made it much less concise. So as long as you run the prompt exactly once…

Does it need to result in compatible code if run by 2 different LLM's? No one complains that Dropbox and Google Drive are incompatible. It would be nice if they were but it hasn't stopped either of them from having lots of use.

Re: Dijkstra On the foolishness of "natural language programming"

#139

Post by [deleted] »

xwiz wrote: post #119
jimmydddd wrote: post #113

Earlier quoted context omitted.

--I think there is a reason why legalese is not plain English This is true. Part of the precision of legalese is that the meanings of some terms have already been more precisely defined by the courts.

This opens an interesting possibility for a purely symbol-based legal code. This would probably improve clarity when it came to legal phrases that overlap common English, and you could avoid ambiguity when it came to language constructs, like in this case[1], where some drivers were losing overtime pay because of a comma in the overtime law. [1] https://cases.justia.com/federal/appellate-courts/ca1/16-190...

[deleted]

Re: Dijkstra On the foolishness of "natural language programming"

#140

Post by alexvitkov »

Computers do what you tell them to, not what you want them to. This is naturally infuriating, as when a computer doesn't do what you want, it means you've failed to express your vague idea in concrete terms.

LLMs in the most general case do neither what you tell them, nor what you want them to. This, surprisingly, can be less infuriating, as now it feels like you have another actor to blame - even though an LLM is still mostly deterministic, and you can get a pretty good idea of what quality of response you can expect for a given prompt.

Post reply on HN