141–150 of 281 posts
Formal can be foolish, too. If you don't believe that, then I have a set for sale, with the property that it contains all sets that don't contain itself.
Earlier quoted context omitted.
I'd recommend learning how transformers work, and the concept of temperature. I don't think I need to cite information that is broadly and readily available, but here: https://medium.com/google-cloud/is-a-zero-temperature-determ... I also qualified the requirement of needing the same hardware, due to FP shenanigans. I could further clarify that you need the same stack (pytorch, tensorflow, etc)
This gcc script that I created below is just as "deterministic" as an LLM. It produces the same result every time. Doesn't make it useful though. echo '#!/usr/bin/env bash' > gcc echo 'cat > gcc openssl rand -base64 100 >> gcc echo 'EOF' >> gcc chmod +x gcc Also, how transformers work is not a spec of the LLM that anyone can use to learn how LLM produces code. It's no gcc source code.
And it is incorrect to base your analysis of future transformer performance on current transformer performance. There is a lot of ongoing research in this area and we have seen continual progress.
Programming is about iteratively expressing a path towards satisfying said goals.
What LLMs are doing now is converting "requirements" into "formalizations".
I don't think Djikstra is wrong in saying - that performing programming in plain-language is a pretty weird idea.
We want to concretize ideas in formalisms. But that's not what any human (including Djikstra) starts with... you start with some sort of goal, some sort of need and requirements.
LLMs merely reduce the time/effort required to go from goals -> formalism.
TLDR: Requirements != Implementation
Please check this talk on the contributions of these mentioned people for the complementary form of deterministic AI (machine intelligence) namely logic, optimization and constraint programming in a seminal lecture by John Hooker [1].
I have got the feeling that if we combine the stochastic nature of LLM based NLP with the deterministic nature of feature structure trchnique based NLP (e.g. CUE), guided by logic, optimization and constraint programming we probably can solve intuitive automation or at least perform proper automation (or automatic computing as Dijkstra put it).
Apparently Yann LeCun also recently proposing optimization based AI namely inference through optimization, or objective driven AI in addition to data-driven AI [2].
Fun facts, you can see Donald Knuth asking questions towards the end of the JH's lecture presentation.
[1] Logic, Optimization, and Constraint Programming: A Fruitful Collaboration - John Hooker - CMU (2023) [video]:
https://www.youtube.com/live/TknN8fCQvRk
[2] Mathematical Obstacles on the Way to Human-Level AI - Yann LeCun - Meta - AMS Josiah Willard Gibbs Lecture at the 2025 Joint Mathematics Meetings (2025) [video]:
Earlier quoted context omitted.
Let’s say that all of the ambiguities are automatically resolved in a reasonable way. This is still not enough to let 2 different computers running two different LLMs to produce compatible code right? And no guarantee of compatibility as you refine it more etc. And if you get into the business of specifying the format/protocol, suddenly you have made it much less concise. So as long as you run the prompt exactly once…
Does it need to result in compatible code if run by 2 different LLM's? No one complains that Dropbox and Google Drive are incompatible. It would be nice if they were but it hasn't stopped either of them from having lots of use.
Earlier quoted context omitted.
The point of LLM is to enable "ordinary people" to write software. This movement is along with "zero code platform", for example. Creating algorithms by drawing block-schemes, by dragging rectangles and arrows. This is old discussion and there are many successful applications of this nature. LLM is just another attempt to tackle this beast. Professional developers don't need this ability indeed. Most professional dev…
I think this is the principle-agent problem at work. Managers/executives who don't understand what programmers do believing that programmers can be easily replaced. Why wouldn't LLM vendors offer to sell it to them? I pity the programmers of the future who will be tasked with maintaining the gargantuan mess these things end up creating.
Earlier quoted context omitted.
Let’s say that all of the ambiguities are automatically resolved in a reasonable way. This is still not enough to let 2 different computers running two different LLMs to produce compatible code right? And no guarantee of compatibility as you refine it more etc. And if you get into the business of specifying the format/protocol, suddenly you have made it much less concise. So as long as you run the prompt exactly once…
Does it need to result in compatible code if run by 2 different LLM's? No one complains that Dropbox and Google Drive are incompatible. It would be nice if they were but it hasn't stopped either of them from having lots of use.
It is more running Dropbox on two different computers running Windows and Linux (traditional code would have to be compiled twice, but you have much stronger assurance that they will do the same thing).
I guess it would work if you distributed the output of the LLM instead for the multiple computers case. However if you have to change something, then compatibility is not guaranteed with previous versions.
Earlier quoted context omitted.
So is more compact better? Does K&R's *d++ = *s++; get a pass now?
When I first read the K&R book, that syntax made perfectly sense. They are building up to it through a few chapters, if I remember correctly. What has changed is that nowadays most developers aren't doing low-level programming anymore, where the building blocks of that expression (or the expression itself) would be common idioms.
/s: that’s because we haven’t gone far enough. People use natural language to generate computer programs. Instead, they should directly run prompts. “You are the graphics system, an entity that manages what is on the screen. You can receive requests from all programs to create and destroys “windows”, and further requests to draw text, lines, circles, etc. in a window created earlier. Items can be of any colour. You a…
Earlier quoted context omitted.
Say, doesn't each business - each activity - have its own formal language? Not as formalized as programming languages, but it's there. Try to define any process, you end up with something trending towards formalized even if you don't realize it.
That's pretty much the whole basis of Domain Driven Design. The core message is to get to an Ubiquitous Language which is the formalization of the business jargon (pretty much a glossary). From which the code can then be derived naturally.
Disagree with "naturally". Unless you want to end up on accidentally quadratic. Or on accidentally exponential if there's such a list.