how did it do regalloc before instruction selection? How do you select the correct register class without knowing which instruction you're gonna use?
We tasked Opus 4.6 using agent teams to build a C Compiler
671–680 of 787 posts
Re: We tasked Opus 4.6 using agent teams to build a C Compiler
#672Re: We tasked Opus 4.6 using agent teams to build a C Compiler
#673Earlier quoted context omitted.
> I can write you an unoptimised C compiler that emits assembly for $20k You may be willing to sell your work at that price, but that’s not the market rate, to put it very mildly. Even 10 times that would be seriously lowballing in the realm of contract work, regardless of whether it’s “optimised” or not (most software isn’t).
> You may be willing to sell your work at that price, but that’s not the market rate, to put it very mildly. It is now. At any rate, this is my actual rate. I live in South Africa, and that's about 4 weeks of work for me, without an AI.
That’s a VERY nice rate for SA; approximately what I charge in the UK. I assume these are not local companies who hire you.
Re: We tasked Opus 4.6 using agent teams to build a C Compiler
#674Earlier quoted context omitted.
I would claim that LLMs desperately need proprietary code in their training, before we see any big gains in quality. There's some incredible source available code out there. Statistically, I think there's a LOT more not so great source available code out there, because the majority of output of seasoned/high skill developers is proprietary. To me, a surprising portion of Claude 4.5 output definitely looks like studen…
I will say many closed source repos are probably equally as poor as open source ones. Even worse in many cases because they are so over engineered nobody understands how they work.
[deleted]
Re: We tasked Opus 4.6 using agent teams to build a C Compiler
#675Earlier quoted context omitted.
> Opus was unable to implement a 16-bit x86 code generator needed to boot into 16-bit real mode. While the compiler can output correct 16-bit x86 via the 66/67 opcode prefixes, the resulting compiled output is over 60kb, far exceeding the 32k code limit enforced by Linux. Instead, Claude simply cheats here and calls out to GCC for this phase Does it really boot...?
The assembler seems like nearly the easiest part. Slurp arch manuals and knock it out, it’s fixed and complete.
I am surprised by the number of comments that say the assembler is trivial - it is admittedly perhaps simpler than some other parts of the compiler chain, but it’s not trivial.
What you are doing is kinda serialising a self-referential graph structure of machine code entries that reference each others addresses, but you don’t know the addresses because the (x86) instructions are variable-length, so you can’t know them until you generate the machine code, chicken-and-egg problem.
Personally I find writing parsers much much simpler than writing assemblers.
Re: We tasked Opus 4.6 using agent teams to build a C Compiler
#676Earlier quoted context omitted.
I'd bet, on average, the quality of proprietary code is worse than open-source code. There have been decades of accumulated slop generated by human agents with wildly varied skill levels, all vibe-coded by ruthless, incompetent corporate bosses.
Not to mention, a team member is (surprise!) fired or let go, and no knowledge transfer exists. Womp, womp. Codebase just gets worse as the organization or team flails. Seen this way too often.
Developers are often treated as cogs. Anyone should be able to step in a pick things up instantly. It’s just typing, right? /s
Re: We tasked Opus 4.6 using agent teams to build a C Compiler
#677Earlier quoted context omitted.
> This test sorta definitely proves that AI is legit. This is an "in distribution" test. There are a lot of C compilers out there, including ones with git history, implemented from scratch. "In distribution" tests do not test generalization. The "out of distribution" test would be like "implement (self-bootstrapping, Linux kernel compatible) C compiler in J." J is different enough from C and I know of no such compile…
There are two compilers that can handle the Linux kernel. GCC and LLVM. Both are written in C, not Rust. It's "in distribution" only if you really stretch the meaning of the term. A generic C compiler isn't going to be anywhere near the level of rigour of this one.
There is tinycc, that makes it three compilers.
There is a C compiler implemented in Rust from scratch: https://github.com/PhilippRados/wrecc/commits/master/?after=... (the very beginning of commit history)
There are several C compilers written in Rust from scratch of comparable quality.
We do not know whether Anthropic has a closed source C compiler written in Rust in their training data. We also do not know whether Anthropic validated their models on their ability to implement C compiler from scratch before releasing this experiment.
That language J I proposed does not have any C compiler implemented in it at all. Idiomatic J expertise is scarce and expensive so that it would be a significant expense for Anthropic to have C compiler in J for their training data. Being Turing-complete, J can express all typical compiler tips and tricks from compiler books, albeit in an unusual way.
Re: We tasked Opus 4.6 using agent teams to build a C Compiler
#678Earlier quoted context omitted.
> The result is a fuzzy reproduction of the training input, specifically of the compilers contained within. Is it? I'm somewhat familiar with gcc and clang's source and it doesn't really particularly look like it to me. https://github.com/anthropics/claudes-c-compiler/blob/main/s... https://llvm.org/doxygen/LoopStrengthReduce_8cpp_source.html https://github.com/gcc-mirror/gcc/blob/master/gcc/gimple-ssa...
Checking for similarity with compilers that consist of orders of magnitudes more code probably doesn't reveal much. There many more smaller compilers for C-adjacent languages out there pkus cod3 fragments from text books.
There are not many more compilers with the specific optimization pass I linked.
Also, I don't think you could reuse code from a different compiler unless you used the same IR.
Re: We tasked Opus 4.6 using agent teams to build a C Compiler
#679It's weird to see the expectation that the result should be perfect. All said and done, that its even possible is remarkable. Maybe these all go into training the next Opus or Sonnet and we start getting models that can create efficient compilers from scratch. That would be something!
This is firmly where I am. "The wonder is not how well the dog dances, it is that it dances at all."
I was also startled when I learned about the human ancestor who was the first to see a mirror.
The brilliance of AI is that it copies(mirrors) imperfectly and you can only look at part_of_the_copy(inference) at a time.
Re: We tasked Opus 4.6 using agent teams to build a C Compiler
#680Earlier quoted context omitted.
> You won't know until you've finished spending the money whether it will fail or not. How do you conclude that? You start off with a bunch of tests and build these things incrementally, why would you spend 20k before realizing there’s a problem?
Because literally no real-world non-research project starts with "we have an extremely comprehensive test suite and specification complete down to the most finite detail" and then searches for a way to turn it into code.
Precisely. Figuring out what the specification is supposed to look like is often the hardest part.