Live data from Hacker News

Dijkstra On the foolishness of "natural language programming"

cs.utexas.edu

121–130 of 281 posts

Re: Dijkstra On the foolishness of "natural language programming"

#121
post #118
post #45

Earlier quoted context omitted.

> Do you find the resulting natural language description is easier to reason about? An example from an different field - aviation weather forecasts and notices are published in a strongly abbreviated and codified form. For example, the weather at Sydney Australia now is: METAR YSSY 031000Z 08005KT CAVOK 22/13 Q1012 RMK RF00.0/000.0 It's almost universal that new pilots ask "why isn't this in words?". And, indeed, mos…

> prefer the coded format. Is is compact... On the other hand "a folder that syncs files between devices and a server" is probably a lot more compact than the code behind Dropbox. I guess you can have both in parallel - prompts and code.

"syncs" can mean so many different things

Re: Dijkstra On the foolishness of "natural language programming"

#122
post #118
post #45

Earlier quoted context omitted.

> Do you find the resulting natural language description is easier to reason about? An example from an different field - aviation weather forecasts and notices are published in a strongly abbreviated and codified form. For example, the weather at Sydney Australia now is: METAR YSSY 031000Z 08005KT CAVOK 22/13 Q1012 RMK RF00.0/000.0 It's almost universal that new pilots ask "why isn't this in words?". And, indeed, mos…

> prefer the coded format. Is is compact... On the other hand "a folder that syncs files between devices and a server" is probably a lot more compact than the code behind Dropbox. I guess you can have both in parallel - prompts and code.

More compact, but also more ambiguous. I suspect an exact specification what Dropbox does in natural language will not be substantially more compact compared to the code.

Re: Dijkstra On the foolishness of "natural language programming"

#123
/s: that’s because we haven’t gone far enough. People use natural language to generate computer programs. Instead, they should directly run prompts.

“You are the graphics system, an entity that manages what is on the screen. You can receive requests from all programs to create and destroys “windows”, and further requests to draw text, lines, circles, etc. in a window created earlier. Items can be of any colour.

You also should send more click information to whomever created the window in which the user clicked the mouse.

There is one special program, the window manager, that can tell you what windows are displayed where on any of the monitors attached to the system”

and

“you are a tic-tac-toe program. There is a graphics system, an entity that manages what is on the screen. You can command it to create and destroys “windows”, and to draw text, lines, circles, etc. in a window created earlier. Items can be of any colour.

The graphics you draw should show a tic-tac-toe game, where users take turn by clicking the mouse. If a user wins the game, it should…

Add ads to the game, unless the user has a pay-per-click subscription”

That should be sufficient to get a game running…

To save it, you’d need another prompt:

”you are a file system, an entity that persists data to disk…”

You also will want

”you are a multi-tasking OS. You give multiple LLMs the idea that they have full control over a system’s CPU and memory. You…”

I look forward to seeing this next year in early April.

Re: Dijkstra On the foolishness of "natural language programming"

#124
post #118
post #45

Earlier quoted context omitted.

> Do you find the resulting natural language description is easier to reason about? An example from an different field - aviation weather forecasts and notices are published in a strongly abbreviated and codified form. For example, the weather at Sydney Australia now is: METAR YSSY 031000Z 08005KT CAVOK 22/13 Q1012 RMK RF00.0/000.0 It's almost universal that new pilots ask "why isn't this in words?". And, indeed, mos…

> prefer the coded format. Is is compact... On the other hand "a folder that syncs files between devices and a server" is probably a lot more compact than the code behind Dropbox. I guess you can have both in parallel - prompts and code.

Let’s say that all of the ambiguities are automatically resolved in a reasonable way.

This is still not enough to let 2 different computers running two different LLMs to produce compatible code right? And no guarantee of compatibility as you refine it more etc. And if you get into the business of specifying the format/protocol, suddenly you have made it much less concise.

So as long as you run the prompt exactly once, it will work, but not necessarily the second time in a compatible way.

Re: Dijkstra On the foolishness of "natural language programming"

#125

Earlier quoted context omitted.

[citation needed] You can set temperature to 0 in many LLMs and get deterministic results (on the same hardware, given floating-point shenanigans). You can provide a well-defined spec and test suite. You can constrain and control the output.

LLMs produce deterministic results ? Now, that 's a big [citation needed]. Where can I find the specs? Edit: This is assuming by "deterministic," you mean the same thing I said about programming language implementations being "controllable, reproducible, and well-defined." If you mean it produces random but same results for the same inputs, then you haven't made any meaningful points.

I'd recommend learning how transformers work, and the concept of temperature. I don't think I need to cite information that is broadly and readily available, but here:

https://medium.com/google-cloud/is-a-zero-temperature-determ...

I also qualified the requirement of needing the same hardware, due to FP shenanigans. I could further clarify that you need the same stack (pytorch, tensorflow, etc)

Re: Dijkstra On the foolishness of "natural language programming"

#126

> It was a significant improvement that now many a silly mistake did result in an error message instead of in an erroneous answer. (And even this improvement wasn't universally appreciated: some people found error messages they couldn't ignore more annoying than wrong results, and, when judging the relative merits of programming languages, some still seem to equate "the ease of programming" with the ease of making un…

Rust? Since when is Rust the pinnacle of static type safety? After I've worked for some time with a language that can express even stronger invariants in types than Rust (Scala) I don't see that property anymore as clear win regardless circumstances . I don't think any more "stronger types == better, no matter what". You have a price to pay for "not being allowed to do mistakes": Explorative work becomes quite diffic…

> You have a price to pay for "not being allowed to do mistakes": Explorative work becomes quite difficult

This is a huge deal for me.

At the beginning of most "what if...?" exercises, I am just trying to get raw tuples of information in and out of some top-level-program logic furnace for the first few [hundred] iterations. I'll likely resort to boxing and extremely long argument lists until what I was aiming for actually takes hold.

I no longer have an urge to define OOP type hierarchies when the underlying domain model is still a vague cloud in my head. When unguided, these abstractions feel like playing Minecraft or Factorio.

Re: Dijkstra On the foolishness of "natural language programming"

#127

People are sticking up for LLMs here and that's cool. I wonder, what if you did the opposite? Take a project of moderate complexity and convert it from code back to natural language using your favorite LLM. Does it provide you with a reasonable description of the behavior and requirements encoded in the source code without losing enough detail to recreate the program? Do you find the resulting natural language descri…

Language can carry tremendous amounts of context. For example:

> I want a modern navigation app for driving which lets me select intersections that I never want to be routed through.

That sentence is low complexity but encodes a massive amount of information. You are probably thinking of a million implementation details that you need to get from that sentence to an actual working app but the opportunity is there, the possibility is there, that that is enough information to get to a working application that solves my need.

And just as importantly, if that is enough to get it built, then “can I get that in cornflower blue instead” is easy and the user can iterate from there.

Re: Dijkstra On the foolishness of "natural language programming"

#128
post #48

Earlier quoted context omitted.

Without descending fully into epistemology, I tend to think that there is no proper "idea" in your head before it's phrased in language - the act of initially describing something in natural language *is* the act of generating it.

Research on LLMs suggest that's probably not the case. See the work on reasoning in latent space, and on shared concepts between languages being represented independently of the individual language. Of course one might argue that even if LLMs are capable of ideation and conceptualisation without natural language, doesn't mean humans are. But the fact that up to 50% of people have no inner monologue seems to refute th…

Humans can't share ideas directly, they pass through language. Some idea is created at the other end, nominally the same one, but there's no reason to suppose a universal internal format. Even translating between human languages loses details, where words in one language map imperfectly to words in another. Moving ideas directly from one brain to another probably can't happen. So the statement in language doesn't map very well to the idea (whichever person's version of the idea). And at some point before the idea is formed there has to be some sort of proto-idea or potential, with less definite boundaries. So "there is no proper idea" sounds right to me. There's something idiosyncratic, not the formal idea that's frozen when put into words.

Re: Dijkstra On the foolishness of "natural language programming"

#130
post #118
post #45

Earlier quoted context omitted.

> Do you find the resulting natural language description is easier to reason about? An example from an different field - aviation weather forecasts and notices are published in a strongly abbreviated and codified form. For example, the weather at Sydney Australia now is: METAR YSSY 031000Z 08005KT CAVOK 22/13 Q1012 RMK RF00.0/000.0 It's almost universal that new pilots ask "why isn't this in words?". And, indeed, mos…

> prefer the coded format. Is is compact... On the other hand "a folder that syncs files between devices and a server" is probably a lot more compact than the code behind Dropbox. I guess you can have both in parallel - prompts and code.

You just cut out half the sentence and responded to one part. Your description is neither well defined nor us it unambiguous.

You can't just pick a singular word out of an argument and argue about that. The argument has a substance, and the substance is not "shorter is better".

Post reply on HN