Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dijkstra On the foolishness of "natural language programming"

cs.utexas.edu

Post reply on HN
Print view View original

221–230 of 281 posts

Re: Dijkstra On the foolishness of "natural language programming"

#221

Post by thaumasiotes »

drpixie wrote: post #45
01100011 wrote: post #24

People are sticking up for LLMs here and that's cool. I wonder, what if you did the opposite? Take a project of moderate complexity and convert it from code back to natural language using your favorite LLM. Does it provide you with a reasonable description of the behavior and requirements encoded in the source code without losing enough detail to recreate the program? Do you find the resulting natural language descri…

> Do you find the resulting natural language description is easier to reason about? An example from an different field - aviation weather forecasts and notices are published in a strongly abbreviated and codified form. For example, the weather at Sydney Australia now is: METAR YSSY 031000Z 08005KT CAVOK 22/13 Q1012 RMK RF00.0/000.0 It's almost universal that new pilots ask "why isn't this in words?". And, indeed, mos…

You can see the same phenomenon playing a roguelike game.

They traditionally have ASCII graphics, and you can easily determine what an enemy is by looking at its ASCII representation.

For many decades now graphical tilesets have been available for people who hate the idea of ASCII graphics. But they have to fit in the same space, and it turns out that it's very difficult to tell what those tiny graphics represent. It isn't difficult at all to identify an ASCII character rendered in one of 16 (?) colors.

Re: Dijkstra On the foolishness of "natural language programming"

#222

Post by xigoi »

tim333 wrote: post #118
drpixie wrote: post #45

Earlier quoted context omitted.

> Do you find the resulting natural language description is easier to reason about? An example from an different field - aviation weather forecasts and notices are published in a strongly abbreviated and codified form. For example, the weather at Sydney Australia now is: METAR YSSY 031000Z 08005KT CAVOK 22/13 Q1012 RMK RF00.0/000.0 It's almost universal that new pilots ask "why isn't this in words?". And, indeed, mos…

> prefer the coded format. Is is compact... On the other hand "a folder that syncs files between devices and a server" is probably a lot more compact than the code behind Dropbox. I guess you can have both in parallel - prompts and code.

I’ll bet my entire net worth that you can’t get an LLM exactly recreate Dropbox from this mescription alone.

Re: Dijkstra On the foolishness of "natural language programming"

#223

Post by fnord77 »

drpixie wrote: post #45
01100011 wrote: post #24

People are sticking up for LLMs here and that's cool. I wonder, what if you did the opposite? Take a project of moderate complexity and convert it from code back to natural language using your favorite LLM. Does it provide you with a reasonable description of the behavior and requirements encoded in the source code without losing enough detail to recreate the program? Do you find the resulting natural language descri…

> Do you find the resulting natural language description is easier to reason about? An example from an different field - aviation weather forecasts and notices are published in a strongly abbreviated and codified form. For example, the weather at Sydney Australia now is: METAR YSSY 031000Z 08005KT CAVOK 22/13 Q1012 RMK RF00.0/000.0 It's almost universal that new pilots ask "why isn't this in words?". And, indeed, mos…

I wonder why the legal profession sticks to natural language

Re: Dijkstra On the foolishness of "natural language programming"

#224

Post by soraminazuki »

soulofmischief wrote: post #142
soraminazuki wrote: post #131

Earlier quoted context omitted.

This gcc script that I created below is just as "deterministic" as an LLM. It produces the same result every time. Doesn't make it useful though. echo '#!/usr/bin/env bash' > gcc echo 'cat > gcc openssl rand -base64 100 >> gcc echo 'EOF' >> gcc chmod +x gcc Also, how transformers work is not a spec of the LLM that anyone can use to learn how LLM produces code. It's no gcc source code.

You claimed they weren't deterministic, I have shown that they can be. I'm not sure what your point is. And it is incorrect to base your analysis of future transformer performance on current transformer performance. There is a lot of ongoing research in this area and we have seen continual progress.

I reiterate:

> This is assuming by "deterministic," you mean the same thing I said about programming language implementations being "controllable, reproducible, and well-defined." If you mean it produces random but same results for the same inputs, then you haven't made any meaningful points.

"Determinism" is a word that you brought up in response to my comment, which I charitably interpreted to mean the same thing I was originally talking about.

Also, it's 100% correct to analyze things based on its fundamental properties. It's absurd to criticize people for assuming 2 + 2 = 4 because "continual progress" might make it 5 in the future.

Re: Dijkstra On the foolishness of "natural language programming"

#225

Post by thaumasiotes »

diputsmonro wrote: post #173
drpixie wrote: post #45

Earlier quoted context omitted.

> Do you find the resulting natural language description is easier to reason about? An example from an different field - aviation weather forecasts and notices are published in a strongly abbreviated and codified form. For example, the weather at Sydney Australia now is: METAR YSSY 031000Z 08005KT CAVOK 22/13 Q1012 RMK RF00.0/000.0 It's almost universal that new pilots ask "why isn't this in words?". And, indeed, mos…

An interesting perspective on this is that language is just another tool on the job. Like any other tool, you use the kind of language that is most applicable and efficient. When you need to describe or understand weather conditions quickly and unambiguously, you use METAR. Sure, you could use English or another natural language, but it's like using a multitool instead of a chef knife. It'll work in a pinch, but a to…

> You can use lots of English words to describe exactly what kabuki is, but if you're going to be talking about it a lot, operating solely in English is going to become burdensome, and it's better to borrow the Japanese word "kabuki".

This is incorrect. Using the word "kabuki" has no advantage over using some other three-syllable word. In both cases you'll be operating solely in English. You could use the (existing!) word "trampoline" and that would be just as efficient. The odds of someone confusing the concepts are low.

Borrowing the Japanese word into English might be easier to learn, if the people talking are already familiar with Japanese, but in the general case it doesn't even have that advantage.

Consider that our name for the Yangtze River is unrelated to the Chinese name of that river. Does that impair our understanding, or use, of the concept?

Re: Dijkstra On the foolishness of "natural language programming"

#226

Post by card_zero »

thaumasiotes wrote: post #225
diputsmonro wrote: post #173

Earlier quoted context omitted.

An interesting perspective on this is that language is just another tool on the job. Like any other tool, you use the kind of language that is most applicable and efficient. When you need to describe or understand weather conditions quickly and unambiguously, you use METAR. Sure, you could use English or another natural language, but it's like using a multitool instead of a chef knife. It'll work in a pinch, but a to…

> You can use lots of English words to describe exactly what kabuki is, but if you're going to be talking about it a lot, operating solely in English is going to become burdensome, and it's better to borrow the Japanese word "kabuki". This is incorrect. Using the word "kabuki" has no advantage over using some other three-syllable word. In both cases you'll be operating solely in English. You could use the (existing!)…

The point is that Japanese has some word for kabuki, while English would have to borrow the word, or coin a new one, or indeed repurpose a word. Without a word, an English speaker would have to resort to a short essay every time the concept was needed, though in practice of course would coin a word quickly.

Hence jargon and formal logic, or something. And surfer slang and txtspk.

Re: Dijkstra On the foolishness of "natural language programming"

#227

Post by staplers »

shit_game wrote: post #172
drpixie wrote: post #45

Earlier quoted context omitted.

> Do you find the resulting natural language description is easier to reason about? An example from an different field - aviation weather forecasts and notices are published in a strongly abbreviated and codified form. For example, the weather at Sydney Australia now is: METAR YSSY 031000Z 08005KT CAVOK 22/13 Q1012 RMK RF00.0/000.0 It's almost universal that new pilots ask "why isn't this in words?". And, indeed, mos…

> Same for maths and coding - once you reach a certain level of expertise, the complexity and redundancy of natural language is a greater cost than benefit. This seems to apply to all fields of expertise. And as well as these points, ambiguity . A formal specification of communication can avoid ambiguity by being absolute and precise regardless of who is speaking and who is interpreting. Natural languages are riddled…

Extending this further, "natural language" changes within populations over time where words or phrases carry different meaning given context. The words "cancel" or "woke" were fairly banal a decade ago. Whereas they can be deeply charged now.

All this to say "natural language"'s best function is interpersonal interaction not defining systems. I imagine most systems thinkers will understand this. Any codified system is essentially its own language.

Re: Dijkstra On the foolishness of "natural language programming"

#228

Post by overfeed »

whatevertrevor wrote: post #219
tim333 wrote: post #116

Earlier quoted context omitted.

Although if you look at most maths textbooks or papers there's a fair bit of English waffle per equation. I guess both have their place.

Textbooks aren't just communicating theorems and proofs (which are often just written in formal symbolic language), but also the language required to teach these concepts, why these are important, how these could be used and sometimes even the story behind the discovery of fields. So this is far from an accurate comparison.

> Textbooks aren't just communicating theorems and proofs

Not even maths papers, which are vehicle for theorem's and proofs, are purely symbolic language and equations. Natural language prose is included when appropriate.

Re: Dijkstra On the foolishness of "natural language programming"

#230

Post by voidhorse »

Dijkstra is entirely correct in this, and it's something I've been trying to urge people to recognize since the beginnings of this LLM wave.

There is inherent value in using formal language to refine, analyze, and describe ideas. This is, after all, why mathematical symbolism has lasted in spite of the fact that all mathematicians are more than capable of talking about mathematical ideas in their natural tongues.

Code realizes a computable model of the world. A computable model is made up of a subset of the mathematical functions we can define. We benefit greatly from formalism in this context. It helps us be precise about the actual relationships and invariants that hold within a system. Stable relationships and invariants lead to predictable behaviors, and predictable systems are reliable systems on the plan of human interaction.

If you describe your system entirely in fuzzily conceived natural language, have you done the requisite analysis to establish the important relationships and invariants among components in your system, or are you just half-assing it?

Engineering is all about establishing relative degrees of certainty in the face of the wild uncertainty that is the default modality of existence. Moving toward a world in which we "engineer" systems increasingly through informal natural language is a step backwards on the continuum of reliability, comprehensibility, and rigor. The fact that anyone considers using these tools and still thinks of themselves as an "engineer" of some kind is an absolute joke.

Post reply on HN