Not if they're made by Anthropic...
Are compilers deterministic?
11–20 of 82 posts
Re: Are compilers deterministic?
#12> . I’m AI-pilled enough to daily-drive comma.ai, and I still want deterministic verification gates around generated code. My girlfriend prefers when I let it drive because it’s smoother and less erratic than I am, which is a useful reminder that “probabilistic system” and “operationally better result” can coexist.
When did the girlfriend enter the discussion? Did I miss something?
Re: Are compilers deterministic?
#13Compilers preserve semantics. That is part of their contract. Whether the output has instructions in one order or another does not matter as long as the output is observationally/functionally equivalent. Article does not do a good job of actually explaining this & instead meanders around sources of irrelevant "stochasticity" like timestamps & build-time UUIDs & concludes by claiming that LLMs have solved the halting…
It matters for things like verifiable builds.
Full determinism is also highly prized by compiler writers because it massively simplifies the task of reproducing and debugging problematic executions.
Re: Are compilers deterministic?
#14I’ve felt like a good response to the vibe coding thing is that customers, product managers, etc ask for features and don’t read the code. You don’t need to read the code of something to build a level of trust about what it does and whether that matches your expectations. It is not that wild that you can have a setup where you get an application and without reading the code decide if it solves your problem to your sa…
There will absolutely be systems of the future that are entirely LLM written. Honestly they will probably be better quality than the standard offshore team output.
But lets all hope these are not vital systems we end up depending on.
Re: Are compilers deterministic?
#15It's not uncommon to have a regression test for compilers that are written in their own language (e.g. some C compilers): compile each new version with itself, then use that to compile itself again, then use the result on unit tests or whatever, which should yield the same results as before.
The point being that determinism of a particular form is expected and required in the instances where they do that.
(I'm not arguing for or against that, I'm simply saying I've seen it in real life projects over the years.)
Re: Are compilers deterministic?
#16> The computer science answer: a compiler is deterministic as a function of its full input state. Engineering answer: most real builds do not control the full input state, so outputs drift. To me that implies the input isn't deterministic, not the compiler itself
You're not wrong but I think the point is to differentiate between the computer science "academic" answer and the engineering "pragmatic" answer. The former is concerned about correctly describing all possible behavior of the compiler, whereas the latter is concerned about what the actual experience is when using the compiler in practice.
You might argue that this is redefining the question in a way that changes the answer, but I'd argue that's also an academic objection; pragmatically, the important thing isn't the exact language but the intent behind the question, and for an engineer being asked this question, it's a lot more likely that the person asking has context for asking that cares about more than just the literal phrasing of "are compilers deterministic?"
Re: Are compilers deterministic?
#17If the output has problems, do you usually rerun the compilation with the same input (that you control)? I don't usually.
What is included in the 'verify' step? Does it involve changing the generated code? If not, how do you ensure things like code quality, architectural constraints, efficiency and consistency? It's difficult, if not (economically) impossible, to write tests for these things. What if the LLM does not follow the guidelines outlined in your prompt? This is still happening. If this is not included, I would call it 'brute forcing'. How much do you pay for tokens?
Re: Are compilers deterministic?
#18> The computer science answer: a compiler is deterministic as a function of its full input state. Engineering answer: most real builds do not control the full input state, so outputs drift. To me that implies the input isn't deterministic, not the compiler itself
> To me that implies the input isn't deterministic, not the compiler itself
or the system upon which the compiler is built (as well as the compiler itself) has made some practical trade offs.
the source file contents are usually deterministic. the order in which they're read and combined and build-time metadata injections often are not (and can be quite difficult to make so).
Re: Are compilers deterministic?
#19> The computer science answer: a compiler is deterministic as a function of its full input state. Engineering answer: most real builds do not control the full input state, so outputs drift. To me that implies the input isn't deterministic, not the compiler itself
You're not wrong but I think the point is to differentiate between the computer science "academic" answer and the engineering "pragmatic" answer. The former is concerned about correctly describing all possible behavior of the compiler, whereas the latter is concerned about what the actual experience is when using the compiler in practice. You might argue that this is redefining the question in a way that changes the…
It matters a lot. For instance, many compilers will put time stamps in their output streams. This can mess up the downstream if your goal is a bit-by-bit identical piece of output across multiple environments.
And that's just one really low hanging fruit type of example, there are many more for instance selecting a different optimization path when memory pressure is high and so on.
Re: Are compilers deterministic?
#20> This comes up now as “is vibecoding sane if LLMs are nondeterministic?” Again: do you want the CS answer, or the engineering answer?
Determinism would help you. With a bit of engineering, you could make LLMs deterministic: basically, fix the random seed for the PRNG and make sure none of the other sources of entropy mentioned earlier in the article contribute.
But that barely impact any of the issues people bring up with LLMs.