[deleted]
Nobody Reviews Compiler Output
21–30 of 36 posts
Re: Nobody Reviews Compiler Output
#22Several days ago I was toying with Rust trying to hack mmtc[0]. Mind you, I have absolutely no idea about Rust, but thanks to the verbosity and helpfulness of its compiler I was able to add a shuffle playlist feature - and am trying to figure out how to add an 'update database' feature too. At some point I even thought learning Rust could be easy...
Re: Nobody Reviews Compiler Output
#23This is why I use LLMs to write directly in Assembly, making it impossible for me to review it. (joke)
Re: Nobody Reviews Compiler Output
#24All other arguments aside... Yes, people do review compiler output, all the time in fact! When optimizing code it's not unusual to look at the assembly. It's not unusual to look for opportunities for autovectorization or to verify inlining or loop unrolling. Compilers are, for the most part, deterministic. This means after people have reviewed the output, it's unlikely to change. It also means if they do change, only…
Not really, people that argue about assembly on Godbolt tend to be the specimens that the site was created for, those that count CPU cycles per Assembly instrution and are religious about which programming language syntax generates the less amount of Assembly opcodes.
The rest of us uses it because it is cool way to share code snippets.
Re: Nobody Reviews Compiler Output
#25Earlier quoted context omitted.
All other arguments aside... Yes, people do review compiler output, all the time in fact! No. In reality, this is almost never done anymore. We used to do it all the time back when performance mattered, but that was then. HN readers don't have to like it, and obviously they (we) don't, but shooting the messenger won't help.
In my circles we're actually doing it even more because we can have an LLM take a look at the assembly.
Your circles are... rarefied.
Re: Nobody Reviews Compiler Output
#26This is a very bad comparison. You can see that immediately if you think to yourself "can I see a future in which we compile code with LLMs?". The answer is no. That's a terrible idea, and the fact that it's a terrible idea is obvious.
The formal foundations of compilers are completely different from the formal foundations of LLMs.
The former are deterministic, easy to formally verify, and extremely simple in nature. "Translate a for-loop into x86 instructions using a set of rules."
The latter is intrinsically statistical in nature. "Translate a human language prompt into functional code" has to infer the correct output statistically from similar, observed input->output relationships. There is no guarantee of consistency. Different builds of the model will see different input->output evidence, in different order, and parameter tuning will further change how it responds to those pieces of evidence. Evidence is incomplete. Local minima are inevitable. LLMs are lossy curve-fitters under the hood. Errors aren't an option, they're an inevitability.
Re: Nobody Reviews Compiler Output
#27""" we need to build: Formal specification layers that agents execute against, not just prompts """ It is probably easier to just write that program.
Right, because to trust that those "formal specifications" are correct, you will have to write them by hand.
First you need to write these specifications and if you say just tell the llm to write them - then how would it be different from just tell the llm to write the program?
I guess you can argue that these are two independent processes so you can combine them to get something more reliable than both - this might be a viable path. But from what I heard writing formal specifications is just really hard - I haven't seen anything practical in this area.
Re: Nobody Reviews Compiler Output
#28I work in embedded and I absolutely review it. If nothing else, for learning.
Re: Nobody Reviews Compiler Output
#29Biggest problem right now is that we have teams pushing dozen of MD files, dozend of thousands of lines of English prose as if they were specs. There’s no way we can validate so much written prose because it’s plain English: perhaps one sentence is written in a way LLMs will read between the lines; perhaps line 100 and line 1000 are contradictory in such a subtle way that LLMs may not be aware of it. In my company we…
What we need is some sort of Common, business-Oriented suBset Of the english Language that can be deterministically translated into something that the machine can understand, but also be read and understood by non-technical stakeholders. Such technology is a pipe dream, but one can dream…
Re: Nobody Reviews Compiler Output
#30I think this argument only holds if you believe that LLMs are at a point where it can handle any combination of craziness that you throw at it.
From my own experience working with agents is that there’s “snowball of shit” effect. Small little mistakes that compound on each other. You can either
- review the code and try to prune some of the shit occasionally - let the LLM handle everything
As of the current status of the industry it’s very hard for me to not see option 2 as extremely irresponsible. Coding agents limits are not well defined and unless you’re running an open weight model locally (most people aren’t) you just gave up all control over your code to a third party. If running local models were the norm, the argument that LLM are just another layer of abstraction would hold a little better. Reusing the compiler analogy from the post, it’s like depending on a compiler where you pay a monthly premium to compile your code. Those did exist a while ago with closed licenses, but I think the majority of deployed code nowadays is on open-ish platforms. This walled garden development paradigm already lost once