Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We tasked Opus 4.6 using agent teams to build a C Compiler

anthropic.com

Post reply on HN
Print view View original

161–170 of 787 posts

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#161

Post by raincole »

geraneum wrote: post #36
NitpickLawyer wrote: post #19

This is a much more reasonable take than the cursor-browser thing. A few things that make it pretty impressive: > This was a clean-room implementation (Claude did not have internet access at any point during its development); it depends only on the Rust standard library. The 100,000-line compiler can build Linux 6.9 on x86, ARM, and RISC-V. It can also compile QEMU, FFmpeg, SQlite, postgres, redis > I started by draf…

> This was a clean-room implementation This is really pushing it, considering it’s trained on… internet, with all available c compilers. The work is already impressive enough, no need for such misleading statements.

It's not a clean-room implementation, but not because it's trained on the internet.

It's not a clean-room implementation because of this:

> The fix was to use GCC as an online known-good compiler oracle to compare against

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#162

Post by chilipepperhott »

rubymamis wrote: post #95
Philpax wrote: post #61

Earlier quoted context omitted.

What Rust-based compiler is it plagiarising from?

There are many, here's a simple Google search: https://github.com/jyn514/saltwater https://github.com/ClementTsang/rustcc https://github.com/maekawatoshiki/rucc

I found this one too: https://github.com/PhilippRados/wrecc

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#163

Post by zamadatix »

Retr0id wrote: post #160
zamadatix wrote: post #159

Earlier quoted context omitted.

Hmm, I didn't have to do that. https://i.imgur.com/OAEtgvr.png But yeah, either way it just needs to know where to find the stdlib.

Probably depends on where your distro puts stuff by default, I think it has a few of the common include paths hardcoded.

Makes sense for the behavior.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#165

Post by ndesaulniers »

I spent a good part of my career (nearly a decade) at Google working on getting Clang to build the linux kernel. https://clangbuiltlinux.github.io/

This LLM did it in (checks notes):

> Over nearly 2,000 Claude Code sessions and $20,000 in API costs

It may build, but does it boot (was also a significant and distinct next milestone)? (Also, will it blend?). Looks like yes!

> The 100,000-line compiler can build a bootable Linux 6.9 on x86, ARM, and RISC-V.

The next milestone is:

Is the generated code correct? The jury is still out on that one for production compilers. And then you have performance of generated code.

> The generated code is not very efficient. Even with all optimizations enabled, it outputs less efficient code than GCC with all optimizations disabled.

Still a really cool project!

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#166

Post by gmueckl »

NitpickLawyer wrote: post #53
gmueckl wrote: post #44

Earlier quoted context omitted.

The result is hardly a clean room implementation. It was rather a brute force attempt to decompress fuzzily stored knowledge contained within the network and it required close steering (using a big suite of tests) to get a reasonable approximation to the desired output. The compression and storage happened during the LLM training. Prove this statement wrong.

> Prove this statement wrong. If all it takes is "trained on the Internet" and "decompress stored knowledge", then surely gpt3, 3.5, 4, 4.1, 4o, o1, o3, o4, 5, 5.1, 5.x should have been able to do it, right? Claude 2, 3, 4, 4.1, 4.5? Surely.

This comparison is only meaningful with comparable numbers of parameters and context window tokens. And then it would mainly test the efficiency and accuracy of the information encoding. I would argue that this is the main improvement over all model generations.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#167

Post by stephc_int13 »

geooff_ wrote: post #158

Maybe I'm naive, but I find these re-engineering complex product posts underwhelming. C Compilers exist and realistically Claudes training corpus contains a ton of C Compiler code. The task is already perfectly defined. There exists a benchmark of well-adopted codebases that can be used to prove if this is a working solution. Half the difficulty in making something is proving it works and is complete. IMO a simpler n…

This is a nice benchmark IMO. I would be curious to see how competitors and improved models would compare.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#168

Post by itay-maman »

My first reaction: wow, incredible.

My second reaction: still incredible, but noting that a C compiler is one of the most rigorously specified pieces of software out there. The spec is precise, the expected behavior is well-defined, and test cases are unambiguous.

I'm curious how well this translates to the kind of work most of us do day-to-day where requirements are fuzzy, many edge cases are discovered on the go, and what we want to build is a moving target.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#169

Post by bs7280 »

geooff_ wrote: post #158

Maybe I'm naive, but I find these re-engineering complex product posts underwhelming. C Compilers exist and realistically Claudes training corpus contains a ton of C Compiler code. The task is already perfectly defined. There exists a benchmark of well-adopted codebases that can be used to prove if this is a working solution. Half the difficulty in making something is proving it works and is complete. IMO a simpler n…

I don't see this as just exercise in making a new useful thing, but benchmarking the SOTA models ability to create a massive* project on its own, with some verifiable metrics of success. I believe they were able to build FFMPEG with this rust compiler?

How much would it cost to pay someone to make a C compiler in rust? A lot more than $20k

* massive meaning "total context needed" >> model context window

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#170

Post by hgs3 »

Rudybega wrote: post #139
falloutx wrote: post #24

So it copied one of the C compilers? This was always possible but now you need to pay $1000 in API costs to Anthropic

It wrote the compiler in Rust. As far as I know, there aren't any Rust based C compilers with the same capabilities. If you can find one that can compile the Linux kernel or get 99% on the GCC torture test suite, I would be quite surprised. I couldn't in a search. Maybe read the article before being so dismissive.

> As far as I know, there aren't any Rust based C compilers with the same capabilities.

If you trained on a neutral representation like an AST or IR, then the source language shouldn't matter. *

* I'm not familiar with how Anthropic builds their models, but training this way should nullify PL differences.

Post reply on HN