Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We tasked Opus 4.6 using agent teams to build a C Compiler

anthropic.com

Post reply on HN
Print view View original

451–460 of 787 posts

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#451

Post by adgjlsfhk1 »

travisgriggs wrote: post #450

A C Compiler seems like one of the more straightforward things to have done. Reading this gives me the same vibe as when a magician does a frequently done trick (saw someone in half, etc). I'd be more interested in letting it have a go at some some of the other "less trodden" paths of computing. Some of the things that would "wow me more": - Build a BEAM alternative, perhaps in an embedded space - Build a Smalltalk V…

if it actually compiles real C correctly, it's pretty impressive. The C standard is a total mess.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#452

Post by esafak »

chamomeal wrote: post #398
NitpickLawyer wrote: post #19

This is a much more reasonable take than the cursor-browser thing. A few things that make it pretty impressive: > This was a clean-room implementation (Claude did not have internet access at any point during its development); it depends only on the Rust standard library. The 100,000-line compiler can build Linux 6.9 on x86, ARM, and RISC-V. It can also compile QEMU, FFmpeg, SQlite, postgres, redis > I started by draf…

What's making these models so much better on every iteration? Is it new data? Different training methods? Kinda waiting for them to plateau so I can stop feeling so existential ¯\_(ツ)_/¯

More compute (bigger models, and prediction-time scaling), algorithmic advances, and ever more data (including synthetic).

Remember that all white collar workers are in your position.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#453

Post by boring-human »

People focused on the flaws are missing the picture. Opus wasn't even trained to be "a member of a team of engineers," it was adapted to the task by one person with a shell script loop. Specific training for this mode of operation is inevitable. And model "IQ" is increasing with every generation. If human IQ is increasing at all, it's only because the engineer pool is shrinking more at one end than the other.

This is a five-alarm fire if you're a SWE and not retiring in the next couple years.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#454

Post by esafak »

echelon wrote: post #442
the8472 wrote: post #295

Earlier quoted context omitted.

"It's like if a squirrel started playing chess and instead of "holy shit this squirrel can play chess!" most people responded with "But his elo rating sucks""

"The squirrel can do my job and more? It can do five years of my work in a month? For only $20k? Pssh, but I bet it copied someone's homework." Developer salaries are about to tank. This is the end of the line. People are just in denial. Soon companies will hire the squirrel instead of you. And the squirrel will transform into enormous infrastructure we can't afford ourselves. "One mega squirrel to implement your own…

IF you want the code to be reviewed and maintained you still need a developer. A developer can craft a better spec.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#455

Post by lelanthran »

p-e-w wrote: post #445
lelanthran wrote: post #441

Earlier quoted context omitted.

> > It cost $20,000 > I'm curious - do you have ANY idea what it costs to have humans write 100,000 lines of code??? I'll bite - I can write you an unoptimised C compiler that emits assembly for $20k, and it won't be 100k lines of code (maybe 15k, the last time I did this?). It won't take me a week, though. I think this project is a good frame of reference and matches my experience - vibing with AI is sometimes more…

> I can write you an unoptimised C compiler that emits assembly for $20k You may be willing to sell your work at that price, but that’s not the market rate, to put it very mildly. Even 10 times that would be seriously lowballing in the realm of contract work, regardless of whether it’s “optimised” or not (most software isn’t).

> You may be willing to sell your work at that price, but that’s not the market rate, to put it very mildly.

It is now.

At any rate, this is my actual rate. I live in South Africa, and that's about 4 weeks of work for me, without an AI.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#456

Post by visarga »

Calavar wrote: post #259
raincole wrote: post #161

Earlier quoted context omitted.

It's not a clean-room implementation, but not because it's trained on the internet. It's not a clean-room implementation because of this: > The fix was to use GCC as an online known-good compiler oracle to compare against

The classical definition of a clean room implementation is something that's made by looking at the output of a prior implementation but not at the source. I agree that having a reference compiler available is a huge caveat though. Even if we completely put training data leakage aside, they're developing against a programmatic checker for a spec that's already had millions of man hours put into it. This is an optimal…

This is the reimplementation scenario for agentic coding. If you have a good spec and battery of tests you can delete the code and reimplement it. Code is no longer the product of eng work, it is more like bytecode now, you regenerate it, you don't read it. If you have to read it then you are just walking a motorcycle.

We have seen at least 3 of these projects - the JustHTML one, the FastRender and this one. All started from beefy tests and specs. They show reimplementation without manual intervention kind of works.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#457

Post by lelanthran »

flakiness wrote: post #446
lelanthran wrote: post #441

Earlier quoted context omitted.

> > It cost $20,000 > I'm curious - do you have ANY idea what it costs to have humans write 100,000 lines of code??? I'll bite - I can write you an unoptimised C compiler that emits assembly for $20k, and it won't be 100k lines of code (maybe 15k, the last time I did this?). It won't take me a week, though. I think this project is a good frame of reference and matches my experience - vibing with AI is sometimes more…

Does it support x64, x8664, arm64 and riscv? (sorry, just trolling - we don't know the quality of backend other than x8664 which is supposed to be able to build bootable linux.)

It's not hard to build a compiler just for a bootable linux.

I see no test criteria that actually runs that built linux through various test plans, so, yeah emitting enough asm just to boot is doable.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#458

Post by ofrzeta »

wmf wrote: post #82
NitpickLawyer wrote: post #25

Earlier quoted context omitted.

It's a bit disappointing that people are still re-hashing the same "it's in the training data" old thing from 3 years ago. It's not like any LLM could 1for1 regurgitate millions of LoC from any training set... This is not how it works. A pertinent quote from the article (which is a really nice read, I'd recommend reading it fully at least once): > Previous Opus 4 models were barely capable of producing a functional c…

In this case it's not reproducing training data verbatim but it probably is using algorithms and data structures that were learned from existing C compilers. On one hand it's good to reuse existing knowledge but such knowledge won't be available if you ask Claude to develop novel software.

You mean like ... a compiler engineer that has learned from books and code samples?

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#459

Post by cowl »

qarl wrote: post #421
bdangubic wrote: post #348

Earlier quoted context omitted.

> On top of that, Anthropic is losing money on it This has got to be my favorite one of them all that keeps coming up in too many comments… You know who also was losing money in the beginning?! every successful company that ever existed! some like Uber were losing billions for a decade. and when was the last time you rode in a taxi? (I still do, my kid never will). not sure how old you are and if you remember “facebo…

I love how your comment is getting downvoted. Like it's a surprise that startups burn through money. I get the feeling that people really have no idea what they're talking about in here anymore. It's a shame.

then you are misunderstaing the downvoting. it's not that the fact that they are burning money. it's the fact that this cost today 20k but that is not the real cost if you factor the it is losing money on this price.

So Tomorrow when this "startup" will need to come out of their money burning phase, like every startup has to sooner or later, that cost will increase, because there is no other monetising avenue, at least not for anthropic that "wilL never use ads".

at 20k this "might" be a reasonable cost for "the project", at 200k it might not.

Re: We tasked Opus 4.6 using agent teams to build a C Compiler

#460

Post by modeless »

LinXitoW wrote: post #407
modeless wrote: post #31

Earlier quoted context omitted.

There seem to still be a lot of people who look at results like this and evaluate them purely based on the current state. I don't know how you can look at this and not realize that it represents a huge improvement over just a few months ago, there have been continuous improvements for many years now, and there is no reason to believe progress is stopping here. If you project out just one year, even assuming progress…

The main issue with improvements in the last year is that a lot of it is based not on the models strictly becoming better, but on tooling being better, and simply using a fuckton more tokens for the same task. Remember that all these companies can only exist because of massive (over)investments in the hope of insane returns and AGI promises. While all these improvements (imho) prove the exact opposite: AGI is absolut…

I disagree. A year ago the models would not come close to doing this, no matter what tools you gave them or how many tokens you generated. Even three months ago. Effectively using tools to complete long tasks required huge improvements in the models themselves. These improvements were driven not by pretraining like before, but by RL with verifiable rewards. This can continue to scale with training compute for the foreseeable future, eliminating the "data wall" we were supposed to be running into.
Post reply on HN