Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We tasked Opus 4.6 using agent teams to build a C Compiler

anthropic.com

Post reply on HN
Print view View original

381–390 of 787 posts

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#381

Post by cryptonector »

geraneum wrote: post #36
NitpickLawyer wrote: post #19

This is a much more reasonable take than the cursor-browser thing. A few things that make it pretty impressive: > This was a clean-room implementation (Claude did not have internet access at any point during its development); it depends only on the Rust standard library. The 100,000-line compiler can build Linux 6.9 on x86, ARM, and RISC-V. It can also compile QEMU, FFmpeg, SQlite, postgres, redis > I started by draf…

> This was a clean-room implementation This is really pushing it, considering it’s trained on… internet, with all available c compilers. The work is already impressive enough, no need for such misleading statements.

Hmm... If Claude iterated a lot then chances are very good that the end result bears little resemblance to open source C compilers. One could check how much resemblance the result actually bears to open source compilers, and I rather suspect that if anyone does check they'll find it doesn't resemble any open source C compiler.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#383

Post by qarl »

bopbopbop7 wrote: post #376
qarl wrote: post #369

Earlier quoted context omitted.

> It cost $20,000 I'm curious - do you have ANY idea what it costs to have humans write 100,000 lines of code??? You should look it up. :)

100k lines of clean, bug free, optimized, and vulnerability free code or 100k lines of outsourced slop? Two very different price points.

A compiler that can build linux.

That level of quality should be sufficient.

Do you know any low quality programmers that write C compilers in rust THAT CAN BUILD LINUX?

No you don't. They do not exist.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#384

Post by cryptonector »

itay-maman wrote: post #168

My first reaction: wow, incredible. My second reaction: still incredible, but noting that a C compiler is one of the most rigorously specified pieces of software out there. The spec is precise, the expected behavior is well-defined, and test cases are unambiguous. I'm curious how well this translates to the kind of work most of us do day-to-day where requirements are fuzzy, many edge cases are discovered on the go, a…

> My second reaction:

This is the key: the more you constrain the LLM, the better it will perform. At least that's my experience with Claude. When working with existing code, the better the code to begin with, the better Claude performs, while if the code has issues then Claude can end up spinning its wheels.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#385

Post by bopbopbop7 »

qarl wrote: post #383
bopbopbop7 wrote: post #376

Earlier quoted context omitted.

100k lines of clean, bug free, optimized, and vulnerability free code or 100k lines of outsourced slop? Two very different price points.

A compiler that can build linux. That level of quality should be sufficient. Do you know any low quality programmers that write C compilers in rust THAT CAN BUILD LINUX? No you don't. They do not exist.

Do you think this was guided by a low quality Anthropic developer?

You can give a developer the GCC test suite and have them build the compiler backwards, which is how this was done. They literally brute forced it, most developers can brute force. It also literally uses GCC in the background... Maybe try reading the article.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#386

Post by Take8435 »

typ wrote: post #364
nomel wrote: post #298

Earlier quoted context omitted.

I would claim that LLMs desperately need proprietary code in their training, before we see any big gains in quality. There's some incredible source available code out there. Statistically, I think there's a LOT more not so great source available code out there, because the majority of output of seasoned/high skill developers is proprietary. To me, a surprising portion of Claude 4.5 output definitely looks like studen…

I'd bet, on average, the quality of proprietary code is worse than open-source code. There have been decades of accumulated slop generated by human agents with wildly varied skill levels, all vibe-coded by ruthless, incompetent corporate bosses.

Not to mention, a team member is (surprise!) fired or let go, and no knowledge transfer exists. Womp, womp. Codebase just gets worse as the organization or team flails.

Seen this way too often.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#387

Post by famouswaffles »

nozzlegear wrote: post #131
modeless wrote: post #31

Earlier quoted context omitted.

There seem to still be a lot of people who look at results like this and evaluate them purely based on the current state. I don't know how you can look at this and not realize that it represents a huge improvement over just a few months ago, there have been continuous improvements for many years now, and there is no reason to believe progress is stopping here. If you project out just one year, even assuming progress…

Every S-curve looks like an exponential until you hit the bend.

Cool I guess. Kind of a meaningless statement yeah? Let's hit the bend, then we'll talk. Until then repeating, 'It's an S Curve guys and what's more, we're near the bend! trust me" ad infinitum is pointless. It's not some wise revelation lol.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#388

Post by D-Machine »

Calavar wrote: post #357
Aurornis wrote: post #262

Earlier quoted context omitted.

Their technique really stretched the definition of extracting text from the LLM. They used a lot of different techniques to prompt with actual text from the book, then asked the LLM to continue the sentences. I only skimmed the paper but it looks like there was a lot of iteration and repetitive trials. If the LLM successfully guessed words that followed their seed, they counted that as "extraction". They had to put i…

Sure, maybe it's tricky to coerce an LLM into spitting out a near verbatim copy of prior data, but that's orthoginal to whether or not the data to create a near verbatim copy exists in the model weights.

Especially since the recalls achieved in the paper are 96% (based on block largest-common substring approaches), the effort of extraction is utterly irrelevant.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#389

Post by qarl »

bopbopbop7 wrote: post #385
qarl wrote: post #383

Earlier quoted context omitted.

A compiler that can build linux. That level of quality should be sufficient. Do you know any low quality programmers that write C compilers in rust THAT CAN BUILD LINUX? No you don't. They do not exist.

Do you think this was guided by a low quality Anthropic developer? You can give a developer the GCC test suite and have them build the compiler backwards, which is how this was done. They literally brute forced it, most developers can brute force. It also literally uses GCC in the background... Maybe try reading the article.

[flagged]

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#390

Post by ajross »

pcloadlett3r wrote: post #372
ajross wrote: post #311

Earlier quoted context omitted.

> $20,000 set on fire To be fair, that's two weeks of the employer cost of a FAANG engineer's labor. And no human hacks a working compiler in two weeks. It's a lot of AI compute for a demo, sure. But $20k stunts are hardly unique. Clearly there's value being demonstrated here.

Is there really value being presented here? Is this codebase a stable enough base to continue developing this compiler or does it warrant a total rewrite? Honest question, it seems like the author mentioned it being at its limits. This mirrors my own experience with Opus in that it isn't that great at defining abstractions in one-shot at least. Maybe with enough loops it could converge but I haven't seen definite pro…

If it generates a booting kernel and passes the test suite at 99% it's probably good enough to use, yeah.

The point isn't to replace GCC per se, it's to demonstrate that reasonably working software of equivalent complexity is within reach for $20k to solve whatever problem it is you do have.

Post reply on HN