LLMs aren't remotely like compilers or power tools
blainehansen.me
LLMs aren't remotely like compilers or power tools
1–10 of 55 posts
Re: LLMs aren't remotely like compilers or power tools
#2[deleted]
Re: LLMs aren't remotely like compilers or power tools
#3This sounds like someone who hasn't used compilers for long. I remember plenty of fun compiler undefined behavior that had essentially non-deterministic outcomes.
And actually using power tools in the real world is pretty uncertain, try running a straight starting cut with a multitool against wood grain and then come back and talk to me.
Re: LLMs aren't remotely like compilers or power tools
#4This sounds like someone who hasn't used compilers for long. I remember plenty of fun compiler undefined behavior that had essentially non-deterministic outcomes. And actually using power tools in the real world is pretty uncertain, try running a straight starting cut with a multitool against wood grain and then come back and talk to me.
Sure, but those were likely bugs in the compiler's codebase that in principle could be isolated and fixed. With an LLM there is no bug and there is no fixing procedure: all outcomes are context-driven and (more or less) nondeterministic by design. I don't mean this in a dismissive way, but there is clearly a difference there.
Re: LLMs aren't remotely like compilers or power tools
#5I've got to give the author credit, it seems like this is written without an LLM. But, there seems to be a lot that is arguable.
For example, the article says "Natural language is the interface of last resort."
But, is that actually true? Isn't the AI boom happening because N/L is truly available as an interface?
Anyway, please keep writing and asking tough questions.
Re: LLMs aren't remotely like compilers or power tools
#6This sounds like someone who hasn't used compilers for long. I remember plenty of fun compiler undefined behavior that had essentially non-deterministic outcomes. And actually using power tools in the real world is pretty uncertain, try running a straight starting cut with a multitool against wood grain and then come back and talk to me.
> And actually using power tools in the real world is pretty uncertain, try running a straight starting cut with a multitool against wood grain and then come back and talk to me.
I especially liked table saws and backhoes as examples of _automation_.
Re: LLMs aren't remotely like compilers or power tools
#7This sounds like someone who hasn't used compilers for long. I remember plenty of fun compiler undefined behavior that had essentially non-deterministic outcomes. And actually using power tools in the real world is pretty uncertain, try running a straight starting cut with a multitool against wood grain and then come back and talk to me.
Another way to think about this: if the compiler has a miscompilation, the user is usually not blamed for the mistake (unless gross lack of QA). If the LLM makes a mistake, the user gets full blame. Thus, slowly combing over the output is required.
Re: LLMs aren't remotely like compilers or power tools
#8The author misses on machine learning compiler optimisation passes, dynamic compilers (aka JIT), profile guided data optimization passes, and that LLM driven compilers is becoming a hot topic in compiler research, see recent SIGPLA or PLDI papers.
Additionally they also miss on low-code/no-code orchestation tools like Boomi, Workato, Opal and co, where programming is moving from drawing boxes, into agent prompting.
Those are also programming from all practical business purposes.
Re: LLMs aren't remotely like compilers or power tools
#9Is there not a way to use a transformer that is deterministic in rounds as it is just maths? I thought it was temperature which randomized results but perhaps it is also something else.
Of course, the content returned is non-deterministic.
Re: LLMs aren't remotely like compilers or power tools
#10This sounds like someone who hasn't used compilers for long. I remember plenty of fun compiler undefined behavior that had essentially non-deterministic outcomes. And actually using power tools in the real world is pretty uncertain, try running a straight starting cut with a multitool against wood grain and then come back and talk to me.
That's not a good counterexample.
Formal languages are called 'formal' because they have rigorously defined semantics. The 'undefined' parts, if anything, reveal that most of the expressible sentences have clearly defined semantics, rather than lack of them.
In addition, the semantics of 'undefined' behaviors is not 'non-deterministic' in the literal sense, since it is, well, undefined.