Live data from Hacker News

Claude Is Not a Compiler

blog.exe.dev

71–80 of 174 posts

Re: Claude Is Not a Compiler

#71
I still think it is (in the source-code generating mode) a compiler, just like LiquidHaskell is. (I haven't used LH but it essentially can automatically supply functions based on the type conditions you specify.)

In my view, reasoning LLMs have learned a large number of close to true sentences of "logic" in natural language. These rules are not consistent (unlike typing rules of LH that allow creation of provably correct programs), but they work in practice better than LH. Essentially, they represent an unsound formal specification of natural language, together with many useful (almost)tautologies that helps solving constraint problems (just like LH does, but again unlike LLM, provably correctly).

LLMs proved that human language can be formalized reasonably close to soundness. I believe we can have a sound formalization of human language, and that kind of formal language will be once superior to LLMs.

It will also not be as complicated as LLMs and require only fraction of compute to use.

Re: Claude Is Not a Compiler

#72
> By the time I was ready to build a keeper, I had accumulated a scar-tissue document that was empirically sufficient to guide an agent through most of the important decisions, at every layer, ranging from high level goals through architecture down to the occasional low level detail, such as the exact shape of the data type for load-bearing concurrent caches.

Waterfall is dead, long live waterfall!

Re: Claude Is Not a Compiler

#73

Earlier quoted context omitted.

>A compiler almost never produces a wrong output back log of compiler's bugs can be pretty large

In a production-grade compiler like LLVM they are rare enough that it compiles giant projects like Linux and Chromium without detectable issues. Maybe eventually Claude can, since it can write tests, but compilers / (deductive) algorithms have other advantages like efficiency and predictability.

I guess we can come back in 20 years and see if Claude is as reliable as LLVM is today. That would give a comparison between similar aged products.

Re: Claude Is Not a Compiler

#74
post #55

LLM-is-a-compiler is indeed a simplistic approach. I wrote a rebuttal to the yesterday's Cursor post, may reuse it here https://replicated.live/blog/follow-up The idea that a 835-page spec "just exists" and we run an LLM to implement it is completely flawed. Specs do not appear out of nowhere, they co-evolve with the code. If you have the code, why do you want to generate it again? Good software is made as a product…

Something I have always (even pre-LLMs) found funny to think about is - all code possible to run on a computer already exists. It's some permutation of all the bits of available memory. It's in there somewhere. So, suppose you want a specific program. 1. Some huge % of those possible programs are obviously not the one you want (most don't even compile). 2. Remaining programs might look similar to the one you want, bu…

all code possible to run on a computer already exists. It's some permutation of all the bits of available memory. It's in there somewhere

This seems either mathematically impossible or vacuous, depending on what you mean ...

Re: Claude Is Not a Compiler

#75

Earlier quoted context omitted.

AI could be deterministic, only it requires sacrificing performance and nobody cares enough. Although algorithms are more predictable , and you can follow an algorithm's steps (even a complex algorithm can be broken into large sequences of steps, which may be broken further).

Determinism is the wrong thing to worry about. As you say an LLM could theoretically be deterministic. Chaos (prompt instability) is the problem that isn’t solvable if you want to treat an LLM like a compiler by doing something like checking in the prompts and rebuilding the code each time. In general if you change a single line of source code, a compiler will not produce a completely different program. If you change…

I think you're correct, I'll just point out that compilers are also chaotic today and it is often accepted. A small change to source program can lead to big change in performance. It's especially apparent in high-level and declarative languages - SQL or anything with GC.

Re: Claude Is Not a Compiler

#76
post #7

> But our VMs start fast, so fast that even if we created the DNS entries before creating the VM, our users still had to sit around waiting for DNS to propagate, which occasionally took minutes, not seconds. To my (limited) understanding, this is not a good idea, and is an unfixable problem from the server side. Companies, VPNs or ISPs or routers, often use their own DNS servers, and those can have caching logic, whi…

Usually when you create the VM, it gets its own DNS subdomain that looks like `my-new-vm-name.exe.xyz`. (I'm not affiliated with exe.dev, I just use their product). So I doubt there would be any cached requests for that particular subdomain before the VM is created.

NXDOMAIN results and other errors have their own TTL value though, so if someone queries too early in the process you could be waiting a little while for their cache to check again. The value for this is set in the SOA record for your DNS zone.

For example, example.org/example.com seems to have TTLs set at 300s (5 minutes) for A records, but the negative caching value is 1800s (30 minutes). If your VM domain is set the same way, you could see a premature lookup causing issues for 30 minutes instead of just 5. Even the 5 minutes could be a pain if your VM spin-up process expects to be able to lookup via the name very early and will fall over if it can't.

Re: Claude Is Not a Compiler

#77
post #55

LLM-is-a-compiler is indeed a simplistic approach. I wrote a rebuttal to the yesterday's Cursor post, may reuse it here https://replicated.live/blog/follow-up The idea that a 835-page spec "just exists" and we run an LLM to implement it is completely flawed. Specs do not appear out of nowhere, they co-evolve with the code. If you have the code, why do you want to generate it again? Good software is made as a product…

In my experience, a spec is often public (or available for purchase by the public from a standards committee for a few hundred dollars, close enough in business terms), but the source code for your competitor's implementations is not available for any price. The first barrier to entry to become a competitor in many industries is to implement an 835 page (note that those are rookie numbers) spec, building software that is compatible with existing data formats or legal frameworks or network protocols or hardware or whatever.

This is another example of the many invisible silos that developers end up in:

1. Some are developing enterprise software for use by their own employer, the spec only exists because it's useful to help the dev team manage the project. In this case, it's totally reasonable to expect the spec to co-evolve with the code. If the release gets pushed out to Q3 or the target has to be adjusted due to unforseen implementation difficulties, so be it.

2. Some are developing contract software for their customers. The spec exists as a communication tool and requirements that allows the developers to know what to build and to inform the customer that what they've built has reached a certain phase of the billing process (including completion). You're on a clock, and if you're late or you don't do what you said you'd do then your customer isn't going to pay you.

3. Some are developing software for abstract customers - either B2B or B2C. Often, the specs here are regulatory or industry-standard, implementing the spec is the function of the business. You're releasing updates and new products to make year-end sales targets or snipe announcements by your competition, and advertising compatibility with various standards is a crucial bullet point in the sales presentation.

Bug reports, feature requests, and especially deadlines have different meanings in each context. The way AI development agents can be used in each silo also varies enormously!

Re: Claude Is Not a Compiler

#78

A compiler is an algorithm and Claude isn't*. A compiler almost never produces a wrong output, even when compiling an extremely complicated program. But a compiler must be clearly defined and is limited to input/commands it's defined for. A compiler will "correctly" process input into unintuitive output, but that's not always what the user intended (e.g. omitting large sections of code that are undefined behavior). C…

If you pair LLMs/AIs like Claude with a formal verification framework you essentially have a new sort of compiler.

Re: Claude Is Not a Compiler

#79
post #63
post #32

Earlier quoted context omitted.

> A compiler is an algorithm and Claude isn't "Algorithm" is not a word with a definition, so I can't say that you're wrong, but I struggle to see how you conceptualize "algorithm" in a way that does include a compiler but does not include a language model. I think the distinction you're trying to draw is better captured by comparing inductive and deductive program synthesis.

> "Algorithm" is not a word with a definition what?

Rogers "Theory of Recursive Functions and Effective Computability", page 1, emphasis is the author's:

§ 1.1 The informal notion of algorithm

In this chapter we give a formal (i.e., mathematically exact) characterization of recursive function. The concept is basic for the remainder of the book. It is one way of making precise the informal mathematical notion of function computable "by algorithm" or "by effective procedure". In this section, as a preliminary to the formal characterization, we discuss certain aspects of the informal notions of algorithm and function computable by algorithm as they occur in mathematics.

I don't understand why you people act like you're stumped by literally, literally page 1 of computer science.

Re: Claude Is Not a Compiler

#80
post #23

Earlier quoted context omitted.

I'm pretty sure they just did...

A rebuttal would spend some time trying to make a case, this is a petulant complaint.

One person's petulant complaint is another person's rebuttal.

I think what you're really saying is you don't agree with OP's perspective...

Post reply on HN