Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We tasked Opus 4.6 using agent teams to build a C Compiler

anthropic.com

Post reply on HN
Print view View original

411–420 of 787 posts

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#411

Post by LinXitoW »

the8472 wrote: post #295
regularfry wrote: post #240

Earlier quoted context omitted.

This is firmly where I am. "The wonder is not how well the dog dances, it is that it dances at all."

"It's like if a squirrel started playing chess and instead of "holy shit this squirrel can play chess!" most people responded with "But his elo rating sucks""

It's more like "We were promised, over and over again, that the squirrel would be autonomous grand master level. We spent insane amounts of money, labour, and opportunity costs of human progress on this. Now, here's a very expensive squirrel, that still needs guidance from a human grandmaster, and most of it's moves are just replications of existing games. Oh, it also can't move the pieces by itself, so it depends on Piece Mover library."

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#412

Post by dcre »

nomel wrote: post #298
jbjbjbjb wrote: post #286

Earlier quoted context omitted.

It’s cool but there’s a good chance it’s just copying someone else’s homework albeit in an elaborate round about way.

I would claim that LLMs desperately need proprietary code in their training, before we see any big gains in quality. There's some incredible source available code out there. Statistically, I think there's a LOT more not so great source available code out there, because the majority of output of seasoned/high skill developers is proprietary. To me, a surprising portion of Claude 4.5 output definitely looks like studen…

This is dead wrong: essentially the entirety of the huge gains in coding performance in the past year have come from RL, not from new sources of training data.

I echo the other commenters that proprietary code isn’t any better, plus it doesn’t matter because when you use LLMs to work on proprietary code, it has the code right there.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#413

Post by LinXitoW »

dwaltrip wrote: post #149
krupan wrote: post #129

Earlier quoted context omitted.

It does lead to poor non-constructive discourse. That's why we keep calling those CEOs to task on it. Why are you not?

The CEOs aren't here in the comments.

Which is why we ought to always bring up their BS every time people try to pretend it didn't happen.

The promises made are ABSOLUTELY relevant to how promising or not these experiments are.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#414

Post by silver_sun »

RyanCavanaugh wrote: post #103
halxc wrote: post #99

Earlier quoted context omitted.

We all saw verbatim copies in the early LLMs. They "fixed" it by implementing filters that trigger rewrites on blatant copyright infringement. It is a research topic for heaven's sake: https://arxiv.org/abs/2504.16046

The internet is hundreds of billions of terabytes; a frontier model is maybe half a terabyte. While they are certainly capable of doing some verbatim recitations, this isn't just a matter of teasing out the compressed C compiler written in Rust that's already on the internet (where?) and stored inside the model.

> this isn't just a matter of teasing out the compressed C compiler written in Rust that's already on the internet (where?)

A quick search brings up several C compilers written in Rust. I'm not claiming they are necessarily in Claude's training data, but they do exist.

https://github.com/PhilippRados/wrecc (unfinished)

https://github.com/ClementTsang/rustcc

https://codeberg.org/notgull/dozer (unfinished)

https://github.com/jyn514/saltwater

I would also like to add that as language models improve (in the sense of decreasing loss on the training set), they in fact become better at compressing their training data ("the Internet"), so that a model that is "half a terabyte" could represent many times more concepts with the same amount of space. Only comparing the relative size of the internet vs a model may not make this clear.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#415

Post by anematode »

GalaxyNova wrote: post #399
andrewshawcare wrote: post #397

It used the best tests it could find for existing compilers. This is effectively steering Claude to a well-defined solution. Hard to find fully specified problems like this in the wild. I think this is more a testament to small, well-written tests than it is agent teams. I imagine you could do the same thing with any frontier model and a single agent in a linear flow. I don’t know why people use parallel agents and i…

> Hard to find fully specified problems like this in the wild. This is such a big and obvious cope. This is obviously a very real problem in the wild and there are many, many others like it. Probably most problems are like this honestly or can be made to be like this.

Impressive, my sarcasm/bait detector almost failed me.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#416

Post by qarl »

sarchertech wrote: post #400
qarl wrote: post #380

Earlier quoted context omitted.

Are you trolling me? Companies (made of humans) write 100,000 LOC all the time. And it's really expensive, despite your suspicions.

No, companies don’t pay people to write 100k LOC. They pay people to write useful software. We figured out that LOC was a useless productivity metric in the 80s.

[flagged]

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#417

Post by pcloadlett3r »

ajross wrote: post #390
pcloadlett3r wrote: post #372

Earlier quoted context omitted.

Is there really value being presented here? Is this codebase a stable enough base to continue developing this compiler or does it warrant a total rewrite? Honest question, it seems like the author mentioned it being at its limits. This mirrors my own experience with Opus in that it isn't that great at defining abstractions in one-shot at least. Maybe with enough loops it could converge but I haven't seen definite pro…

If it generates a booting kernel and passes the test suite at 99% it's probably good enough to use, yeah. The point isn't to replace GCC per se, it's to demonstrate that reasonably working software of equivalent complexity is within reach for $20k to solve whatever problem it is you do have.

> it's probably good enough to use, yea.

Not for general purpose use, only for demo.

> that reasonably working software of equivalent complexity is within reach for $20k to solve

But if this can't come close to replacing GCC and can't be modified without introducing bugs then it hasn't proven this yet. I learned some new hacks from the paper and that's great and all but from my experiencing of trying to harness even 4 claude sessions in parallel on a complex task it just goes off the rails in terms of coherence. I'll try the new techniques but my intuition is that its not really as good as you are selling it.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#418

Post by svieira »

bdangubic wrote: post #379
PostOnce wrote: post #365

Earlier quoted context omitted.

Are we forgetting that sometimes, they just go bankrupt?

name one with comparable number of users and revenue? not saying you are wrong but I would bet against the outcome

Enron

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#419

Post by [deleted] »

sarchertech wrote: post #400
qarl wrote: post #380

Earlier quoted context omitted.

Are you trolling me? Companies (made of humans) write 100,000 LOC all the time. And it's really expensive, despite your suspicions.

No, companies don’t pay people to write 100k LOC. They pay people to write useful software. We figured out that LOC was a useless productivity metric in the 80s.

[deleted]

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#420

Post by charcircuit »

qarl wrote: post #361
kvemkon wrote: post #355

Earlier quoted context omitted.

> optimizations aren't as good as the 40 year gcc project with all optimizations disabled: > Even with all optimizations enabled, it outputs less efficient code than GCC with all optimizations disabled .

That distinction doesn't change my point. I am not surprised that a 40 year old project generates better code than this brand new one.

Not only is it new. There has been 0 performance optimization done. Well none prompted for at least. Once you give the agents a profiler and start a loop focusing on performance you'll see it start improving it.
Post reply on HN