461–470 of 787 posts
This is more an example of code distribution rather than intelligence. If Claude had NOT been trained on compiler code, it would NOT have been able to build a compiler. Definitely signals the end of software IP or at least in its present form.
I spent a good part of my career (nearly a decade) at Google working on getting Clang to build the linux kernel. https://clangbuiltlinux.github.io/ This LLM did it in (checks notes): > Over nearly 2,000 Claude Code sessions and $20,000 in API costs It may build, but does it boot (was also a significant and distinct next milestone)? (Also, will it blend?). Looks like yes! > The 100,000-line compiler can build a bootab…
> Still a really cool project! Yeah. This test sorta definitely proves that AI is legit. Despite the millions of people still insisting it's a hoax. The fact that the optimizations aren't as good as the 40 year gcc project? Eh - I think people who focus on that are probably still in some serious denial.
> This test sorta definitely proves that AI is legit.
The "out of distribution" test would be like "implement (self-bootstrapping, Linux kernel compatible) C compiler in J." J is different enough from C and I know of no such compiler.
I spent a good part of my career (nearly a decade) at Google working on getting Clang to build the linux kernel. https://clangbuiltlinux.github.io/ This LLM did it in (checks notes): > Over nearly 2,000 Claude Code sessions and $20,000 in API costs It may build, but does it boot (was also a significant and distinct next milestone)? (Also, will it blend?). Looks like yes! > The 100,000-line compiler can build a bootab…
> Still a really cool project! Yeah. This test sorta definitely proves that AI is legit. Despite the millions of people still insisting it's a hoax. The fact that the optimizations aren't as good as the 40 year gcc project? Eh - I think people who focus on that are probably still in some serious denial.
Earlier quoted context omitted.
That distinction doesn't change my point. I am not surprised that a 40 year old project generates better code than this brand new one.
Not only is it new. There has been 0 performance optimization done. Well none prompted for at least. Once you give the agents a profiler and start a loop focusing on performance you'll see it start improving it.
When you are optimizing a program, you have a specific part of code to improve. The part can be found with profiler.
When you are optimizing a compiler generated code, you have many similar parts of code in many programs and not-so-specific part of compiler that can be improved.
Earlier quoted context omitted.
The classical definition of a clean room implementation is something that's made by looking at the output of a prior implementation but not at the source. I agree that having a reference compiler available is a huge caveat though. Even if we completely put training data leakage aside, they're developing against a programmatic checker for a spec that's already had millions of man hours put into it. This is an optimal…
This is the reimplementation scenario for agentic coding. If you have a good spec and battery of tests you can delete the code and reimplement it. Code is no longer the product of eng work, it is more like bytecode now, you regenerate it, you don't read it. If you have to read it then you are just walking a motorcycle. We have seen at least 3 of these projects - the JustHTML one, the FastRender and this one. All star…
JustHTML is a success in large part because it's a problem that can be solved with 4 digit LOC. The whole codebase can sit in an LLM's context at once. Do LLMs scale beyond that?
I would classify both FastRender and Opus C compiler as interesting failures. They are interesting because they got a non-negligible fraction of the way to feature complete. They are failures because they ended with no clear path for moving the needle forward to 80% feature complete, let alone 100%.
From the original article:
> The resulting compiler has nearly reached the limits of Opus’s abilities. I tried (hard!) to fix several of the above limitations but wasn’t fully successful. New features and bugfixes frequently broke existing functionality.
From the experiments we've seen so far it seems that a large enough agentic code base will inevitably collapse under its own weight.
Earlier quoted context omitted.
Not only is it new. There has been 0 performance optimization done. Well none prompted for at least. Once you give the agents a profiler and start a loop focusing on performance you'll see it start improving it.
We are talking about compiler here and "performance" referred above is the performance of generated code. When you are optimizing a program, you have a specific part of code to improve. The part can be found with profiler. When you are optimizing a compiler generated code, you have many similar parts of code in many programs and not-so-specific part of compiler that can be improved.
I spent a good part of my career (nearly a decade) at Google working on getting Clang to build the linux kernel. https://clangbuiltlinux.github.io/ This LLM did it in (checks notes): > Over nearly 2,000 Claude Code sessions and $20,000 in API costs It may build, but does it boot (was also a significant and distinct next milestone)? (Also, will it blend?). Looks like yes! > The 100,000-line compiler can build a bootab…
Real usable AI would create it with simple: 'make c compilers c99 faster than GCC'.
AI usage should be banned in general. It takes jobs faster than creating new ones ..